Papers
Topics
Authors
Recent
Search
2000 character limit reached

Towards Open-set Gesture Recognition via Feature Activation Enhancement and Orthogonal Prototype Learning

Published 5 Dec 2023 in cs.CV | (2312.02535v1)

Abstract: Gesture recognition is a foundational task in human-machine interaction (HMI). While there has been significant progress in gesture recognition based on surface electromyography (sEMG), accurate recognition of predefined gestures only within a closed set is still inadequate in practice. It is essential to effectively discern and reject unknown gestures of disinterest in a robust system. Numerous methods based on prototype learning (PL) have been proposed to tackle this open set recognition (OSR) problem. However, they do not fully explore the inherent distinctions between known and unknown classes. In this paper, we propose a more effective PL method leveraging two novel and inherent distinctions, feature activation level and projection inconsistency. Specifically, the Feature Activation Enhancement Mechanism (FAEM) widens the gap in feature activation values between known and unknown classes. Furthermore, we introduce Orthogonal Prototype Learning (OPL) to construct multiple perspectives. OPL acts to project a sample from orthogonal directions to maximize the distinction between its two projections, where unknown samples will be projected near the clusters of different known classes while known samples still maintain intra-class similarity. Our proposed method simultaneously achieves accurate closed-set classification for predefined gestures and effective rejection for unknown gestures. Extensive experiments demonstrate its efficacy and superiority in open-set gesture recognition based on sEMG.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (42)
  1. Z. Sun, S. Wang, C. Liu, and X. Ma, “Metaphoraction: Support gesture-based interaction design with metaphorical meanings,” ACM Transactions on Computer-Human Interaction, vol. 29, pp. 1–33, 2022.
  2. A. Carfì and F. Mastrogiovanni, “Gesture-based human–machine interaction: Taxonomy, problem definition, and analysis,” IEEE Transactions on Cybernetics, vol. 53, pp. 497–513, 2023.
  3. C. Han, S. Xiang, and D. Qian, “Mtdl-net: Morphological and temporal discriminative learning for heartbeat classification,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).   IEEE, 2023, pp. 1–5.
  4. D. S. Copaci, J. Arias, M. Gómez-Tome, L. Moreno, and D. Blanco, “semg-based gesture classifier for a rehabilitation glove,” Front. Neurorobot., vol. 16, p. 750482, 2022.
  5. Q. Liu, X. Wu, Y. Jiang, X. Liu, Y. Zhang, X. Xu, and L. Qi, “A fully connected deep learning approach to upper limb gesture recognition in a secure FES rehabilitation environment,” Int. J. Intell. Syst., vol. 36, no. 5, pp. 2387–2411, 2021.
  6. D. Xiong, D. Zhang, X. Zhao, and Y. Zhao, “Deep learning for emg-based human-machine interaction: A review,” IEEE CAA J. Autom. Sinica, vol. 8, no. 3, pp. 512–533, 2021.
  7. W. Lin, C. Li, and Y. Zhang, “Interactive application of data glove based on emotion recognition and judgment system,” Sensors, vol. 22, no. 17, p. 6327, 2022.
  8. P. Gao, “Key technologies of human-computer interaction for immersive somatosensory interactive games using VR technology,” Soft Comput., vol. 26, no. 20, pp. 10 947–10 956, 2022.
  9. J. Kim, S. Yang, B. Koo, S. Lee, S. Park, S. Kim, K. Cho, and Y. Kim, “semg-based hand posture recognition and visual feedback training for the forearm amputee,” Sensors, vol. 22, no. 20, p. 7984, 2022.
  10. P. Tsinganos, B. Cornelis, J. Cornelis, B. Jansen, and A. Skodras, “Improved gesture recognition based on semg signals and TCN,” in ICASSP, 2019, pp. 1169–1173.
  11. N. K. Karnam, S. R. Dubey, A. Turlapaty, and B. Gokaraju, “Emghandnet: A hybrid cnn and bi-lstm architecture for hand activity classification using surface emg signals,” Biocyb. Biomed. Eng., 2022.
  12. N. Mendes, “Surface electromyography signal recognition based on deep learning for human-robot interaction and collaboration,” J. Intell. Robotic Syst., vol. 105, no. 2, p. 42, 2022.
  13. S. Xiang, Y. Fu, M. Xie, Z. Yu, and T. Liu, “Unsupervised person re-identification by hierarchical cluster and domain transfer,” Multimedia Tools and Applications, vol. 79, pp. 19 769–19 786, 2020.
  14. S. Xiang, D. Qian, M. Guan, B. Yan, T. Liu, Y. Fu, and G. You, “Less is more: Learning from synthetic data with fine-grained attributes for person re-identification,” ACM Transactions on Multimedia Computing, Communications and Applications, vol. 19, no. 5s, pp. 1–20, 2023.
  15. W. J. Scheirer, A. de Rezende Rocha, A. Sapkota, and T. E. Boult, “Toward open set recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 7, pp. 1757–1772, 2013.
  16. H. Yang, X. Zhang, F. Yin, Q. Yang, and C. Liu, “Convolutional prototype network for open set recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 5, pp. 2358–2370, 2022.
  17. G. Chen, P. Peng, X. Wang, and Y. Tian, “Adversarial reciprocal points learning for open set recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 11, pp. 8065–8081, 2022.
  18. Z. Xia, P. Wang, G. Dong, and H. Liu, “Adversarial kinetic prototype framework for open set recognition.” IEEE Trans. Neural Netw. Learn. Syst., vol. PP, 2023.
  19. H. Cevikalp, B. Uzun, Y. Salk, H. Saribas, and O. Köpüklü, “From anomaly detection to open set recognition: Bridging the gap,” Pattern Recognit., vol. 138, p. 109385, 2023.
  20. T. G. Dietterich and A. Guyer, “The familiarity hypothesis: Explaining the behavior of deep open set methods,” Pattern Recognit., vol. 132, p. 108931, 2022.
  21. K. Park and S. Lee, “Movement intention decoding based on deep learning for multiuser myoelectric interfaces,” in BCI, 2016, pp. 1–2.
  22. M. Atzori, A. Gijsberts, C. Castellini, B. Caputo, A.-G. M. Hager, S. Elsig, G. Giatsidis, F. Bassetto, and H. Müller, “Electromyography data for non-invasive naturally-controlled robotic hand prostheses,” Sci. Data, vol. 1, 2014.
  23. L. Wu, X. Chen, X. Chen, and X. Zhang, “Rejecting novel motions in high-density myoelectric pattern recognition using hybrid neural networks,” Front. Neurorobot., vol. 16, p. 862193, 2022.
  24. L. Wu, A. Liu, X. Zhang, X. Chen, and X. Chen, “Unknown motion rejection in myoelectric pattern recognition using convolutional prototype network,” IEEE Sens. J., vol. 22, pp. 4305–4314, 2022.
  25. A. Bendale and T. E. Boult, “Towards open set deep networks,” in CVPR, 2016, pp. 1563–1572.
  26. D. Zhou, H. Ye, and D. Zhan, “Learning placeholders for open-set recognition,” in CVPR, 2021, pp. 4401–4410.
  27. P. Perera, V. I. Morariu, R. Jain, V. Manjunatha, C. Wigington, V. Ordonez, and V. M. Patel, “Generative-discriminative feature representations for open-set recognition,” in CVPR, 2020, pp. 11 811–11 820.
  28. H. Huang, Y. Wang, Q. Hu, and M. Cheng, “Class-specific semantic reconstruction for open set recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 4, pp. 4214–4228, 2023.
  29. W. Bao, Q. Yu, and Y. Kong, “Evidential deep learning for open set action recognition,” in ICCV, 2021, pp. 13 329–13 338.
  30. W. Moon, J. H. Park, H. S. Seong, C. Cho, and J. Heo, “Difficulty-aware simulator for open set recognition,” in ECCV, vol. 13685, 2022, pp. 365–381.
  31. L. Neal, M. L. Olson, X. Z. Fern, W. Wong, and F. Li, “Open set learning with counterfactual images,” in ECCV, vol. 11210, 2018, pp. 620–635.
  32. A. R. Dhamija, M. Günther, and T. E. Boult, “Reducing network agnostophobia,” in NeurIPS, 2018, pp. 9175–9186.
  33. W. Cho and J. Choo, “Towards accurate open-set recognition via background-class regularization,” in ECCV, vol. 13685, 2022, pp. 658–674.
  34. C. Liu and M. Nakagawa, “Evaluation of prototype learning algorithms for nearest-neighbor classifier in application to handwritten character recognition,” Pattern Recognit., vol. 34, no. 3, pp. 601–615, 2001.
  35. W. Wang, W. Pei, Q. Cao, S. Liu, G. Lu, and Y. Tai, “Push for center learning via orthogonalization and subspace masking for person re-identification,” IEEE Trans. Image Process., vol. 30, pp. 907–920, 2021.
  36. Y. Shen, X. Sun, and X. Wei, “Equiangular basis vectors,” in CVPR, 2023, pp. 11 755–11 765.
  37. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in CVPR, 2016, pp. 770–778.
  38. S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Comput., vol. 9, no. 8, pp. 1735–1780, 1997.
  39. X. Li, W. Wang, X. Hu, and J. Yang, “Selective kernel networks,” in CVPR, 2019, pp. 510–519.
  40. M. Ortiz-Catalan, R. Brånemark, and B. Håkansson, “Biopatrec: A modular research platform for the control of artificial limbs based on pattern recognition algorithms,” Source Code Biol. Medicine, vol. 8, p. 11, 2013.
  41. S. Pizzolato, L. Tagliapietra, M. Cognolato, M. Reggiani, H. Müller, and M. Atzori, “Comparison of six electromyography acquisition setups on hand movement classification tasks,” PLoS ONE, vol. 12, 2017.
  42. A. Krasoulis, I. Kyranou, M. S. Erden, K. Nazarpour, and S. Vijayakumar, “Improved prosthetic hand control with concurrent use of myoelectric and inertial measurements,” J. Neuroeng. Rehabil., vol. 14, 2017.
Citations (2)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up