A Feedback-Control Framework for Efficient Dataset Collection from In-Vehicle Data Streams
Abstract: Modern AI systems are increasingly constrained not by model capacity but by the quality and diversity of their data. Despite growing emphasis on data-centric AI, most datasets are still gathered in an open-loop manner which accumulates redundant samples without feedback from the current coverage. This results in inefficient storage, costly labeling, and limited generalization. To address this, this paper introduces \ac{FCDC}, a paradigm that formulates data collection as a closed-loop control problem. \ac{FCDC} continuously approximates the state of the collected data distribution using an online probabilistic model and adaptively regulates sample retention using based on feedback signals such as likelihood and Mahalanobis distance. Through this feedback mechanism, the system dynamically balances exploration and exploitation, maintains dataset diversity, and prevents redundancy from accumulating over time. Besides showcasing the controllability of \ac{FCDC} on a synthetic dataset, experiments on a real data stream show that \ac{FCDC} produces more balanced datasets by $\SI{25.9}{\percent}$ while reducing data storage by $\SI{39.8}{\percent}$. These results demonstrate that data collection itself can be actively controlled, transforming collection from a passive pipeline stage into a self-regulating, feedback-driven process at the core of data-centric AI.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Explain it Like I'm 14
Overview
This paper is about a smarter way to collect data from cars with cameras and sensors. Instead of saving everything (which wastes space and time), the authors design a system that “listens” to what’s already in the dataset and uses feedback to decide which new pieces of data are worth keeping. They call this idea FCDC, short for “Feedback-Controlled Data Collection.”
What is the paper trying to do?
In simple terms, the paper asks:
- How can we collect the right kind of data (not just a lot of data) so AI systems learn better?
- Can we make the data collection process adjust itself in real time, based on what has already been collected?
- Will this reduce repeated (redundant) data and keep the dataset more balanced and diverse?
How did they do it? (Methods explained simply)
Think of the dataset like a garden, and each new data sample (like a car image) is a seed. If you keep planting the same kind of seed, your garden isn’t very diverse. The system in this paper watches what’s already planted and decides which seeds to plant next to keep the garden balanced.
Technically, here’s the loop they use, with simple analogies:
- Embedding function (
phi): This turns a raw data sample (like an image) into a simple, measurable description. For example, “How many cars are in this image?” It’s like summarizing a book by counting chapters. - Dataset estimator (
E): This keeps an up-to-date guess of what the dataset looks like. It tracks the “center” and “spread” of the data (like a bell curve’s average and width). Think of it as a librarian who knows which kinds of books are already on the shelf. - Value function (
psi): This calculates how useful a new sample is based on what’s missing or overrepresented. It’s like a score: high scores mean “this sample adds something new,” low scores mean “we’ve seen too many like this.”- They use signals such as:
- Likelihood: How typical is this sample compared to the data we already have?
- Mahalanobis distance: A smarter “how far from the center” measure that accounts for the shape of the data. Imagine measuring distance not just with a ruler, but with a ruler that bends to fit the data’s shape.
- Collection control (
F): This is the final decision—keep or discard the sample—like a gatekeeper that says yes or no in real time.
A helpful analogy: It works like a thermostat. A thermostat compares current temperature to your target temperature and adjusts heating accordingly. Here, the system compares the current dataset to a target dataset shape (for example, “evenly spread across different types of scenes”) and adjusts what to collect next.
They test two simple “value functions” (ways to score samples):
- Redundancy reduction: Down-weight very common samples (don’t keep too many of the same).
- Uniform target: Try to collect samples so the dataset looks evenly balanced across a chosen region or feature (e.g., across the number of cars seen).
All updates happen online, meaning the car decides on the fly—very important because cars produce massive data (up to 2.5 GB per second) and can’t store everything.
What did they find?
They ran two kinds of experiments:
- Synthetic (simulated) data:
- They generated simple 2D data shaped like a bell curve and showed the system can “steer” collection toward different goals—either avoid dense areas (redundancy) or collect more evenly (uniform).
- This proves the approach is controllable: changing the value function changes what the dataset looks like.
- Real car data:
- They used a 20-minute stream of 1,356 images from a test vehicle.
- The goal: collect a dataset that’s balanced by “number of vehicles per image” (so you don’t just have scenes with 8–11 cars over and over).
- Results:
- FCDC reduced stored samples by about 39.8% (less data saved, but smarter choices).
- It improved dataset balance by about 25.9% (measured with the “coefficient of variation,” where lower is better).
- The open-loop (save-everything) approach kept too many similar scenes; FCDC saved fewer but more diverse scenes. Rare scenes stayed rare (because the world doesn’t produce many of them), but overrepresented scenes were kept in check.
Why are these results important?
- Better data, not just more data: AI models increasingly need high-quality, varied datasets. Saving everything leads to lots of duplicates and higher labeling/storage costs.
- Real-time control: This shows that data collection can be actively controlled as it happens, not just cleaned up later.
- Resource-friendly: Cars and edge devices have limited storage and compute. FCDC uses feedback and simple updates to make efficient choices without constant retraining.
What does this mean for the future?
This approach shifts data collection from “dump everything” to “curate as you go.” It helps:
- Cut storage and labeling costs.
- Build datasets that are more representative of the real world, improving AI performance.
- React to changes in the environment (like new kinds of scenes) by collecting smarter.
Next steps could include:
- Handling many features at once (not just “number of cars”), like lighting, weather, or object distances.
- Tying the value function directly to model performance (collect what helps the model most).
- Detecting and adjusting for distribution shifts (when the world changes, the collector adapts).
In short: FCDC is like putting a brain into the data pipeline. It decides, in real time, what to keep, so AI learns from the most useful and diverse information.
Glossary
- Active learning: A learning paradigm that selects which data points to label to improve model performance efficiently. Example: "Active learning focuses on selecting which samples to label, not which to collect and relies on computationally intensive model retraining."
- Adaptive controller: A control component that adjusts its behavior based on feedback to meet desired objectives under changing conditions. Example: "integrates a probabilistic density estimator with an adaptive controller to regulate sampling based on real-time feedback of novelty and redundancy."
- Anomaly detection: Techniques for identifying unusual or rare events/data that deviate from expected patterns. Example: "Another approach for vehicle data collection is anomaly detection, which is systematized in different levels~\cite{20_Breitenstein}."
- Closed-loop control: A control approach that continuously uses feedback from the system’s output to adjust its inputs in real time. Example: "formulates data collection as a closed-loop control problem."
- Coefficient of variation: A normalized measure of dispersion defined as the ratio of the mean to the standard deviation (as defined in the paper). Example: "The balance of the dataset throughout the collection process is monitored using the coefficient of variation, defined as $CV(\mathcal{D})= \sfrac{\mu}{\sigma}$."
- Control policy: A decision rule mapping system state and inputs to control actions. Example: "Consequently, a control policy"
- Continual learning: Methods that enable models to adapt to new data over time without retraining from scratch, while mitigating forgetting. Example: "Continual learning adapts model parameters rather than managing incoming data itself."
- Coreset selection: Techniques that choose a representative subset of data that preserves performance for training or evaluation. Example: "Coreset selection methods operate in static batches,"
- Data-centric AI: An approach emphasizing the quality, diversity, and management of data as the primary driver of AI performance. Example: "Despite growing emphasis on data-centric AI, most datasets are still gathered in an open-loop manner"
- Discrete-time nonlinear stochastic dynamics: A mathematical description of systems that evolve in discrete steps with nonlinear relationships and randomness. Example: "The resulting discrete-time nonlinear stochastic dynamics are given by:"
- Distribution shift: A change in the data distribution between training and deployment that can degrade model performance. Example: "leverage the feedback structure to monitor and counteract distribution shifts."
- Exogenous disturbance: An external input that influences a system but is not controlled by it. Example: "The incoming data stream is viewed as an exogenous disturbance"
- Fast Data paradigm: A data handling approach emphasizing real-time, iterative processing under resource constraints. Example: "Fast Data paradigm, requiring real-time and iterative updates with minimal computational overhead."
- Feedback Control Data Collection (FCDC): A framework that treats data collection as a controllable process using feedback to regulate what data to retain. Example: "Data Flow Schematic of the Feedback Control Data Collection framework."
- Ledoit–Wolf shrinkage: A covariance estimation technique that improves conditioning by shrinking the sample covariance toward a structured target. Example: "a covariance estimation using Ledoit-Wolf shrinkage"
- Long-tail data distribution: A distribution where many rare events occur infrequently but collectively represent a significant portion of the data. Example: "the phenomenon of the long-tail data distribution"
- Mahalanobis distance: A distance measure that accounts for correlations in the data by using the covariance matrix. Example: "Mahalanobis distance."
- Mahalanobis ellipse: The set of points at a fixed Mahalanobis distance from the mean, forming an ellipse in 2D. Example: "we define as a Mahalanobis ellipse,"
- Open-loop data collection: Collecting data without feedback from the current dataset state, leading to potential redundancy and bias. Example: "A comparison between the data distributions obtained through the \ac{FCDC} and an open-loop data collection strategy is presented"
- Operational Design Domain (ODD): The set of conditions under which an automated system is intended to operate. Example: "Operational Design Domain in the automotive context."
- Oracle Approximating Shrinkage: A method for estimating covariance matrices by approximating an optimal (oracle) shrinkage level. Example: "Oracle Approximating Shrinkage~\cite{10715246} estimator"
- Probabilistic density estimator: A model that estimates the probability distribution of data, often used for measuring novelty or redundancy. Example: "integrates a probabilistic density estimator with an adaptive controller"
- Q–Q diagram: A plot comparing the quantiles of two distributions to assess how similar they are. Example: "Q-Q Diagram of the data collection strategies random collection"
- Value function: A function that assigns a utility or priority to candidate samples, guiding selection decisions. Example: "the Value Function is computed to update the collection strategy"
- Welford's algorithm: An online algorithm for numerically stable computation of streaming mean and variance. Example: "updated incrementally from each observed sample using Welford's algorithm~\cite{Welford1962NoteOA}"
- YOLO-based object detection: Object detection using the YOLO (You Only Look Once) architecture for fast, real-time detection. Example: "first, a YOLO-based object detection is applied to each incoming image"
Collections
Sign up for free to add this paper to one or more collections.