Continual Learning from Simulated Interactions via Multitask Prospective Rehearsal for Bionic Limb Behavior Modeling
Abstract: Lower limb amputations and neuromuscular impairments severely restrict mobility, necessitating advancements beyond conventional prosthetics. While motorized bionic limbs show promise, their effectiveness depends on replicating the dynamic coordination of human movement across diverse environments. In this paper, we introduce a model for human behavior in the context of bionic prosthesis control. Our approach leverages human locomotion demonstrations to learn the synergistic coupling of the lower limbs, enabling the prediction of the kinematic behavior of a missing limb during tasks such as walking, climbing inclines, and stairs. We propose a multitasking, continually adaptive model that anticipates and refines movements over time. At the core of our method is a technique called multitask prospective rehearsal, that anticipates and synthesizes future movements based on the previous prediction and employs a corrective mechanism for subsequent predictions. Our evolving architecture merges lightweight, task-specific modules on a shared backbone, ensuring both specificity and scalability. We validate our model through experiments on real-world human gait datasets, including transtibial amputees, across a wide range of locomotion tasks. Results demonstrate that our approach consistently outperforms baseline models, particularly in scenarios with distributional shifts, adversarial perturbations, and noise.
- M. Windrich, M. Grimmer, O. Christ, S. Rinderknecht, and P. Beckerle, “Active lower limb prosthetics: a systematic review of design issues and solutions,” Biomedical engineering online, vol. 15, no. 3, p. 140, 2016.
- B. E. Lawson, Control methodologies for powered prosthetic interventions in unilateral and bilateral transfemoral amputees. PhD thesis, Vanderbilt University, 2014.
- G. Chen, Z. Liu, L. Chen, and P. Yang, “Control of powered knee joint prosthesis based on finite-state machine,” in Proceedings of the 2015 Chinese Intelligent Automation Conference, pp. 395–403, Springer, 2015.
- B. E. Lawson, A. H. Shultz, and M. Goldfarb, “Evaluation of a coordinated control system for a pair of powered transfemoral prostheses,” in 2013 IEEE International Conference on Robotics and Automation, pp. 3888–3893, IEEE, 2013.
- N. Dhir, H. Dallali, E. M. Ficanha, G. A. Ribeiro, and M. Rastgaar, “Locomotion envelopes for adaptive control of powered ankle prostheses,” in 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 1488–1495, IEEE, 2018.
- E. V. Zabre-Gonzalez, L. Riem, P. A. Voglewede, B. Silver-Thorn, S. R. Koehler-McNicholas, and S. A. Beardsley, “Continuous myoelectric prediction of future ankle angle and moment across ambulation conditions and their transitions,” Frontiers in Neuroscience, vol. 15, p. 709422, 2021.
- B. Fang, Q. Zhou, F. Sun, J. Shan, M. Wang, C. Xiang, and Q. Zhang, “Gait neural network for human-exoskeleton interaction,” Frontiers in Neurorobotics, vol. 14, 2020.
- A. Kumar, J. Hong, A. Singh, and S. Levine, “Should i run offline reinforcement learning or behavioral cloning?,” in International Conference on Learning Representations, 2022.
- S. Ross and D. Bagnell, “Efficient reductions for imitation learning,” in Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 661–668, 2010.
- M. M. Ardestani, Z. Chen, L. Wang, Q. Lian, Y. Liu, J. He, D. Li, and Z. Jin, “Feed forward artificial neural network to predict contact force at medial knee joint: Application to gait modification,” Neurocomputing, vol. 139, pp. 114–129, 2014.
- H. Zhang, Y. Guo, and D. Zanotto, “Accurate ambulatory gait analysis in walking and running using machine learning models,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 28, no. 1, pp. 191–202, 2019.
- V. Rai, A. Sharma, and E. Rombokas, “Mode-free control of prosthetic lower limbs,” in 2019 International Symposium on Medical Robotics (ISMR), pp. 1–7, IEEE, 2019.
- Y. Wang, Z. Li, Y. Chen, W. Liao, and A. Wang, “Human gait prediction for lower limb rehabilitation exoskeleton using gated recurrent units,” in RiTA 2020: Proceedings of the 8th International Conference on Robot Intelligence Technology and Applications, pp. 128–135, Springer, 2021.
- C. Livolsi, R. Conti, F. Giovacchini, N. Vitiello, and S. Crea, “A novel wavelet-based gait segmentation method for a portable hip exoskeleton,” IEEE Transactions on Robotics, vol. 38, no. 3, pp. 1503–1517, 2021.
- M. Kim and L. J. Hargrove, “A gait phase prediction model trained on benchmark datasets for evaluating a controller for prosthetic legs,” Frontiers in Neurorobotics, vol. 16, p. 1064313, 2023.
- A. Rattanasak, P. Uthansakul, M. Uthansakul, T. Jumphoo, K. Phapatanaburi, B. Sindhupakorn, and S. Rooppakhun, “Real-time gait phase detection using wearable sensors for transtibial prosthesis based on a knn algorithm,” Sensors, vol. 22, no. 11, p. 4242, 2022.
- S. Dey, T. Yoshida, R. H. Foerster, M. Ernst, T. Schmalz, R. M. Carnier, and A. F. Schilling, “A hybrid approach for dynamically training a torque prediction model for devising a human-machine interface control strategy,” arXiv preprint arXiv:2110.03085, 2021.
- J. L. Waugh, E. Huang, J. E. Fraser, K. B. Beyer, A. Trinh, W. E. McIlroy, and D. Kulić, “Online learning of gait models from older adult data,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 27, no. 4, pp. 733–742, 2019.
- M. McCloskey and N. J. Cohen, “Catastrophic interference in connectionist networks: The sequential learning problem,” in Psychology of learning and motivation, vol. 24, pp. 109–165, Elsevier, 1989.
- R. M. French, “Catastrophic forgetting in connectionist networks,” Trends in cognitive sciences, vol. 3, no. 4, pp. 128–135, 1999.
- J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, et al., “Overcoming catastrophic forgetting in neural networks,” Proceedings of the national academy of sciences, vol. 114, no. 13, pp. 3521–3526, 2017.
- F. Zenke, B. Poole, and S. Ganguli, “Continual learning through synaptic intelligence,” in International Conference on Machine Learning, pp. 3987–3995, PMLR, 2017.
- A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, K. Kavukcuoglu, R. Pascanu, and R. Hadsell, “Progressive neural networks,” arXiv preprint arXiv:1606.04671, 2016.
- D. Lopez-Paz and M. Ranzato, “Gradient episodic memory for continual learning,” Advances in neural information processing systems, vol. 30, 2017.
- G. Merlin, V. Lomonaco, A. Cossu, A. Carta, and D. Bacciu, “Practical recommendations for replay-based continual learning methods,” in International Conference on Image Analysis and Processing, pp. 548–559, Springer, 2022.
- V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing atari with deep reinforcement learning,” arXiv preprint arXiv:1312.5602, 2013.
- D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, et al., “Mastering the game of go with deep neural networks and tree search,” nature, vol. 529, no. 7587, pp. 484–489, 2016.
- S. Racanière, T. Weber, D. Reichert, L. Buesing, A. Guez, D. Jimenez Rezende, A. Puigdomènech Badia, O. Vinyals, N. Heess, Y. Li, et al., “Imagination-augmented agents for deep reinforcement learning,” Advances in neural information processing systems, vol. 30, 2017.
- A. Nagabandi, G. Kahn, R. S. Fearing, and S. Levine, “Neural network dynamics for model-based deep reinforcement learning with model-free fine-tuning,” in 2018 IEEE international conference on robotics and automation (ICRA), pp. 7559–7566, IEEE, 2018.
- K. Chua, R. Calandra, R. McAllister, and S. Levine, “Deep reinforcement learning in a handful of trials using probabilistic dynamics models,” Advances in neural information processing systems, vol. 31, 2018.
- W. Zhou, S. Bajracharya, and D. Held, “Plas: Latent action space for offline reinforcement learning,” in Conference on Robot Learning, pp. 1719–1735, PMLR, 2021.
- T. Yu, G. Thomas, L. Yu, S. Ermon, J. Y. Zou, S. Levine, C. Finn, and T. Ma, “Mopo: Model-based offline policy optimization,” Advances in Neural Information Processing Systems, vol. 33, pp. 14129–14142, 2020.
- P. Englert, A. Paraschos, M. P. Deisenroth, and J. Peters, “Probabilistic model-based imitation learning,” Adaptive Behavior, vol. 21, no. 5, pp. 388–403, 2013.
- R. Kidambi, J. Chang, and W. Sun, “Mobile: Model-based imitation learning from observation alone,” Advances in Neural Information Processing Systems, vol. 34, pp. 28598–28611, 2021.
- A. Hu, G. Corrado, N. Griffiths, Z. Murez, C. Gurau, H. Yeo, A. Kendall, R. Cipolla, and J. Shotton, “Model-based imitation learning for urban driving,” Advances in Neural Information Processing Systems, vol. 35, pp. 20703–20716, 2022.
- A. v. d. Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu, “Wavenet: A generative model for raw audio,” arXiv preprint arXiv:1609.03499, 2016.
- G. M. van de Ven, H. T. Siegelmann, and A. S. Tolias, “Brain-inspired replay for continual learning with artificial neural networks,” Nature communications, vol. 11, no. 1, pp. 1–14, 2020.
- B. Hu, E. Rouse, and L. Hargrove, “Benchmark datasets for bilateral lower-limb neuromechanical signals from wearable sensors during unassisted locomotion in able-bodied individuals,” Frontiers in Robotics and AI, vol. 5, p. 14, 2018.
- K. R. Embry, D. J. Villarreal, R. L. Macaluso, and R. D. Gregg, “Modeling the kinematics of human locomotion over continuously varying speeds and inclines,” IEEE transactions on neural systems and rehabilitation engineering, vol. 26, no. 12, pp. 2342–2350, 2018.
- D. M. Endres and J. E. Schindelin, “A new metric for probability distributions,” IEEE Transactions on Information theory, vol. 49, no. 7, pp. 1858–1860, 2003.
- C. Prahm, A. Schulz, B. Paaßen, J. Schoisswohl, E. Kaniusas, G. Dorffner, B. Hammer, and O. Aszmann, “Counteracting electrode shifts in upper-limb prosthesis control via transfer learning,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 27, no. 5, pp. 956–962, 2019.
- I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” arXiv preprint arXiv:1412.6572, 2014.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.