Improve training stability and sample efficiency of online continual learning algorithms

Improve the training stability and sample efficiency of online learning algorithms used for continual learning in embodied robotics, ensuring reliable convergence and effective learning under non-stationary, real-world data streams.

Background

Online continual learning is essential for robots operating in changing environments, but current algorithms often suffer from instability and high sample requirements, limiting practical deployment.

The authors explicitly list improved stability and sample efficiency of online algorithms as an open research problem, highlighting the need for methods that can robustly learn from streaming, non-i.i.d. data typical of embodied scenarios.

References

Open research problems and viable approaches include: 1) mixing different proportions of prior data distribution when fine-tuning on the latest data to alleviate catastrophic forgetting , 2) developing efficient prototypes from prior distributions or curricula for task inference in learning new tasks, 3) improving training stability and sample efficiency of online learning algorithms, 4) identifying principled ways to seamlessly incorporate large-capacity models into control frameworks, potentially through hierarchical learning or slow-fast control, for real-time inference.

— Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI  (2407.06886 - Liu et al., 2024) in Section 8, Challenges and Future Directions – Continual Learning