Value Prediction for Spatiotemporal Gait Data Using Deep Learning
Abstract: Human gait has been commonly used for the diagnosis and evaluation of medical conditions and for monitoring the progress during treatment and rehabilitation. The use of wearable sensors that capture pressure or motion has yielded techniques that analyze the gait data to aid recovery, identify activity performed, or identify individuals. Deep learning, usually employing classification, has been successfully utilized in a variety of applications such as computer vision, biomedical imaging analysis, and natural language processing. We expand the application of deep learning to value prediction of time-series of spatiotemporal gait data. Moreover, we explore several deep learning architectures (Recurrent Neural Networks (RNN) and RNN combined with Convolutional Neural Networks (CNN)) to make short- and long-distance predictions using two different experimental setups. Our results show that short-distance prediction has an RMSE as low as 0.060675, and long-distance prediction RMSE as low as 0.106365. Additionally, the results show that the proposed deep learning models are capable of predicting the entire trial when trained and validated using the trials from the same participant. The proposed, customized models, used with value prediction open possibilities for additional applications, such as fall prediction, in-home progress monitoring, aiding of exoskeleton movement, and authentication.
- Gait analysis: clinical facts. European Journal of Physical and Rehabilitation Medicine, 52(4):560–574, 2016.
- Mobility lab by apdm wearable technologies. URL https://apdm.com/mobility/.
- Design of a biofeedback device for gait rehabilitation in post-stroke patients. In Proc. IEEE 58th International Midwest Symposium on Circuits and Systems (MWSCAS), pages 1–4, 2015. doi:https://doi.org/10.1109/mwscas.2015.7282097.
- A mobile gait monitoring system for gait analysis. In 2009 IEEE International Conference on Rehabilitation Robotics, pages 73–79, 2009. doi:10.1109/ICORR.2009.5209621.
- Detection of abnormalities in a human gait using smart shoes. Proceedings of SPIE - The International Society for Optical Engineering, 6932, 05 2008. doi:10.1117/12.776003.
- A hidden markov model-based technique for gait segmentation using a foot-mounted gyroscope. In In Proceedings of Annual International Conference of the IEEE Engineering in Medicine and Biology Society., pages 4369–73, 08 2011. doi:10.1109/IEMBS.2011.6091084.
- Machine learning based human gait segmentation with wearable sensor platform. In 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pages 588–594, 2019. doi:10.1109/EMBC.2019.8857509.
- Multimodal few-shot learning for gait recognition. Applied Sciences, 10(21), 2020a. ISSN 2076-3417. doi:10.3390/app10217619. URL https://www.mdpi.com/2076-3417/10/21/7619.
- Weedgait: Unobtrusive smartphone sensing of marijuana-induced gait impairment by fusing gait cycle segmentation and neural networks. In 2019 IEEE Healthcare Innovations and Point of Care Technologies, (HI-POCT), pages 91–94, 2019. doi:10.1109/HI-POCT45284.2019.8962787.
- Common sport activity recognition using inertial sensor. 2018 IEEE 14th International Colloquium on Signal Processing & Its Applications (CSPA), pages 67–71, 2018.
- Classification of gait type based on deep learning using various sensors with smart insole. Sensors, 19(8), 2019a. ISSN 1424-8220. URL https://www.mdpi.com/1424-8220/19/8/1757.
- Learning to assess the quality of stroke rehabilitation exercises. In Proceedings of the 24th International Conference on Intelligent User Interfaces, IUI ’19, page 218–228, New York, NY, USA, 2019b. Association for Computing Machinery. ISBN 9781450362726. doi:10.1145/3301275.3302273. URL https://doi-org.csulb.idm.oclc.org/10.1145/3301275.3302273.
- Learning a predictive model of human gait for the control of a lower-limb exoskeleton. In In Proceedings of 5th IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics (BIOROB 2014), 08 2014. doi:10.1109/BIOROB.2014.6913830.
- Fall prediction in neurological gait disorders: differential contributions from clinical assessment, gait analysis, and daily-life mobility monitoring. Journal of Neurology, 03 2021. doi:10.1007/s00415-021-10504-x.
- A practical gait analysis system using gyroscopes. Medical Engineering and Physics, 21(2):87–94, 1999. ISSN 1350-4533. doi:https://doi.org/10.1016/S1350-4533(99)00030-2. URL https://www.sciencedirect.com/science/article/pii/S1350453399000302.
- Assessment of walking features from foot inertial sensing. IEEE Transactions on Biomedical Engineering, 52(3):486–494, 2005. doi:10.1109/TBME.2004.840727.
- Jay Hertel Rachel M Koldenhoven. Validation of a wearable sensor for measuring running biomechanics. Digit Biomark, 2:74–78, 08 2018. doi:10.1159/000491645.
- Estimation of running injury risks using wearable sensors. In ISBS Proceedings Archive, 36 (1), pages 240–243, February 2018. URL https://commons.nmu.edu/isbs/vol36/iss1/30.
- Real-time biofeedback device for gait rehabilitation of post-stroke patients. Biomedical Engineering Letters, 7, 06 2017. doi:10.1007/s13534-017-0036-1.
- Gait phase analysis based on a hidden markov model. Mechatronics, 21(6):961–970, 2011. ISSN 0957-4158. doi:https://doi.org/10.1016/j.mechatronics.2011.03.003. URL https://www.sciencedirect.com/science/article/pii/S0957415811000481.
- Deep Learning in Gait Analysis for Security and Healthcare, pages 299–334. Springer International Publishing, Cham, 2020. ISBN 978-3-030-31760-7. doi:10.1007/978-3-030-31760-7_10. URL https://doi.org/10.1007/978-3-030-31760-7_10.
- Deep learning for sensor-based activity recognition: A survey. Pattern Recognition Letters, 119:3–11, 2019. ISSN 0167-8655. doi:https://doi.org/10.1016/j.patrec.2018.02.010. URL https://www.sciencedirect.com/science/article/pii/S016786551830045X. Deep Learning for Pattern Recognition.
- Activity classification using realistic data from wearable sensors. IEEE Transactions on Information Technology in Biomedicine, 10(1):119–128, 2006. doi:10.1109/TITB.2005.856863.
- Footlogger. URL http://www.footlogger.com/.
- Can ensemble deep learning identify people by their gait using data collected from multi-modal sensors in their insole? Sensors, 20(14), 2020b. ISSN 1424-8220. doi:10.3390/s20144001. URL https://www.mdpi.com/1424-8220/20/14/4001.
- Accurate ambulatory gait analysis in walking and running using machine learning models. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 28(1):191–202, 2020. doi:10.1109/TNSRE.2019.2958679.
- Evaluation of deep convolutional neural networks for detection of freezing of gait in parkinson’s disease patients. Biomedical Signal Processing and Control, 46:221–230, 2018.
- A deep learning approach to human activity recognition based on single accelerometer. 2015 IEEE International Conference on Systems, Man, and Cybernetics, pages 1488–1492, 2015.
- Explaining the unique nature of individual gait patterns with deep learning. Scientific Reports, 9, 2019.
- Wearable imu for shoulder injury prevention in overhead sports. Sensors, 16(11), 2016. ISSN 1424-8220. doi:10.3390/s16111847. URL https://www.mdpi.com/1424-8220/16/11/1847.
- Human motion trajectory prediction: A survey. Int. J. Rob. Res., 39(8):895–935, jul 2020. ISSN 0278-3649. doi:10.1177/0278364920917446. URL https://doi.org/10.1177/0278364920917446.
- Intent prediction of pedestrians via motion trajectories using stacked recurrent neural networks. IEEE Transactions on Intelligent Vehicles, 3(4):414–424, 2018. doi:10.1109/TIV.2018.2873901.
- Motion prediction of traffic actors for autonomous driving using deep convolutional networks. CoRR, abs/1808.05819, 2018. URL http://arxiv.org/abs/1808.05819.
- Ss-lstm: A hierarchical lstm model for pedestrian trajectory prediction. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 1186–1194, 2018. doi:10.1109/WACV.2018.00135.
- Multi-agent tensor fusion for contextual trajectory prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12126–12134, 2019.
- Estimating movements of human body for the shirt-type wearable device mounted on the strain sensors based on convolutional neural networks. In 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pages 5871–5876, 2019. doi:10.1109/EMBC.2019.8856722.
- A survey of human gait-based artificial intelligence applications. Frontiers in Robotics and AI, 8, 2022. ISSN 2296-9144. doi:10.3389/frobt.2021.749274. URL https://www.frontiersin.org/articles/10.3389/frobt.2021.749274.
- Latest research trends in gait analysis using wearable sensors and machine learning: A systematic review. IEEE Access, 8:167830–167864, 2020.
- Research of fall detection and fall prevention technologies: A systematic review. IEEE Access, 7:77702–77722, 2019. doi:10.1109/ACCESS.2019.2922708.
- Biological motion during inflammation in humans. Brain, Behavior, and Immunity, 84:147–153, 2020. ISSN 0889-1591. doi:https://doi.org/10.1016/j.bbi.2019.11.019. URL https://www.sciencedirect.com/science/article/pii/S0889159119306427.
- Activity recognition and abnormal behaviour detection with recurrent neural networks. Procedia Computer Science, 110:86–93, 2017. ISSN 1877-0509. doi:https://doi.org/10.1016/j.procs.2017.06.121. URL https://www.sciencedirect.com/science/article/pii/S1877050917313005. 14th International Conference on Mobile Systems and Pervasive Computing (MobiSPC 2017) / 12th International Conference on Future Networks and Communications (FNC 2017) / Affiliated Workshops.
- Moving the lab into the mountains: A pilot study of human activity recognition in unstructured environments. Sensors, 21(2), 2021. ISSN 1424-8220. doi:10.3390/s21020654. URL https://www.mdpi.com/1424-8220/21/2/654.
- Human activity recognition from sensor-based large-scale continuous monitoring of parkinson’s disease patients. 2017 IEEE/ACM International Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE), pages 249–250, 2017.
- A machine learning approach for gait speed estimation using skin-mounted wearable sensors: From healthy controls to individuals with multiple sclerosis. PLoS One, 2017. doi:doi:10.1371/journal.pone.0178366.
- Wearable device-based gait recognition using angle embedded gait dynamic images and a convolutional neural network. Sensors, 17(3), 2017. ISSN 1424-8220. doi:10.3390/s17030478. URL https://www.mdpi.com/1424-8220/17/3/478.
- Machine learning and deep learning. Electron Markets, 31:685–695, 2021. doi:https://doi.org/10.1007/s12525-021-00475-2.
- Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
- Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097–1105, 2012.
- Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1–9, 2015.
- Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
- A field guide to dynamical recurrent networks. John Wiley & Sons, 2001.
- On the difficulty of training recurrent neural networks. In International conference on machine learning, pages 1310–1318, 2013.
- Understanding the exploding gradient problem. CoRR, abs/1211.5063, 2:417, 2012.
- Visualizing and understanding recurrent networks. arXiv preprint arXiv:1506.02078, 2015.
- A survey on theories and applications for self-driving cars based on deep learning methods. Applied Sciences, 10(8), 2020. ISSN 2076-3417. doi:10.3390/app10082749. URL https://www.mdpi.com/2076-3417/10/8/2749.
- Faster r-cnn: Towards real-time object detection with region proposal networks. arXiv preprint arXiv:1506.01497, 2015.
- Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017.
- Attention is all you need. arXiv preprint arXiv:1706.03762, 2017.
- Deep encoder–decoder neural networks for retinal blood vessels dense prediction. International Journal of Computational Intelligence Systems, 14(1):1078–1086, 2021.
- Deep convolutional neural networks for multi-modality isointense infant brain image segmentation. NeuroImage, 108:214–224, 2015.
- Gradient amplification: An efficient way to train deep neural networks. Big Data Mining and Analytics, 3(3):196–207, 2020. doi:10.26599/BDMA.2020.9020004.
- Long short-term memory. Neural computation, 9(8):1735–1780, 1997.
- Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11):2673–2681, 1997.
- Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
- Machine learning for predicting stock market movement using news headlines. In 2020 IEEE Green Energy and Smart Systems Conference (IGESSC), pages 1–6, 2020. doi:10.1109/IGESSC50231.2020.9285163.
- Forecasting stock prices from the limit order book using convolutional neural networks. In 2017 IEEE 19th Conference on Business Informatics (CBI), volume 01, pages 7–12, 2017. doi:10.1109/CBI.2017.23.
- Neural abstractive text summarization with sequence-to-sequence models. ACM Transactions on Data Science, 2(1):1–37, 2021.
- Adam optimizer keras. URL https://keras.io/api/optimizers/adam/.
- Rmsprop optimizer keras. URL https://keras.io/api/optimizers/rmsprop/.
- Differentiable neural network pruning to enable smart applications on microcontrollers. 6(4), jan 2023. doi:10.1145/3569468. URL https://doi.org/10.1145/3569468.
- Age-related differences in healthy adults walking patterns under a cognitive task with deep neural networks. Frontiers in Bioengineering and Biotechnology, 8:260, 2020. doi:https://doi.org/10.3389/fbioe.2020.00260.
- A deep learning approach to emg-based classification of gait phases during level ground walking. Electronics, 8(8):894, Aug 2019. ISSN 2079-9292. doi:10.3390/electronics8080894. URL http://dx.doi.org/10.3390/electronics8080894.
- Age-related differences in healthy adults walking patterns under a cognitive task with deep neural networks. IEEE Sensors Journal, 21(2):2353–2363, 2021. doi:10.1109/JSEN.2020.3021349.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.