2000 character limit reached
Towards EMG-to-Speech with a Necklace Form Factor
Published 31 Jul 2024 in eess.AS | (2407.21345v1)
Abstract: Electrodes for decoding speech from electromyography (EMG) are typically placed on the face, requiring adhesives that are inconvenient and skin-irritating if used regularly. We explore a different device form factor, where dry electrodes are placed around the neck instead. 11-word, multi-speaker voiced EMG classifiers trained on data recorded with this device achieve 92.7% accuracy. Ablation studies reveal the importance of having more than two electrodes on the neck, and phonological analyses reveal similar classification confusions between neck-only and neck-and-face form factors. Finally, speech-EMG correlation experiments demonstrate a linear relationship between many EMG spectrogram frequency bins and self-supervised speech representation dimensions.
- D. Gaddy and D. Klein, “An improved model for voicing silent speech,” in ACL-IJCNLP, 2021.
- J. S. Galego, O. V. Casas, D. Rossato, A. Simões, and A. Balbinot, “Surface electromyography and electroencephalography processing in dysarthric patients for verbal commands or speaking intention characterization,” Measurement, vol. 175, p. 109147, 2021.
- C. E. Stepp, J. T. Heaton, R. G. Rolland, and R. E. Hillman, “Neck and face surface electromyography for prosthetic voice control after total laryngectomy,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 17, no. 2, p. 146–155, Apr. 2009. [Online]. Available: http://dx.doi.org/10.1109/TNSRE.2009.2017805
- F. Wang and E. M.-L. Yiu, “Surface electromyographic activity of the suprahyoid and sternocleidomastoid muscles in pitch and loudness control,” Frontiers in Physiology, vol. 14, May 2023.
- A. D. Chan, K. Englehart, B. Hudgins, and D. F. Lovely, “Myo-electric signals to augment speech recognition,” Medical and Biological Engineering and Computing, 2001.
- G. S. Meltzner, J. T. Heaton, Y. Deng, G. De Luca, S. H. Roy, and J. C. Kline, “Silent speech recognition as an alternative communication device for persons with laryngectomy,” TASLP, 2017.
- S.-C. Jou, L. Maier-Hein, T. Schultz, and A. Waibel, “Articulatory feature classification using surface electromyography,” in ICASSP, 2006.
- J. Lian, C. Feng, N. Farooqi, S. Li, A. Kashyap, C. J. Cho, P. Wu, R. Netzorg, T. Li, and G. K. Anumanchipalli, “Unconstrained dysfluency modeling for dysfluent speech transcription and detection,” in ASRU, 2023.
- M. Lyu, C. Xiong, and Q. Zhang, “Electromyography based chinese voice command recognition,” in 2014 IEEE International Conference on Information and Automation (ICIA). IEEE, Jul. 2014.
- J. Wu, Y. Zhang, L. Xie, Y. Yan, X. Zhang, S. Liu, X. An, E. Yin, and D. Ming, “A novel silent speech recognition approach based on parallel inception convolutional neural network and mel frequency spectral coefficient,” Frontiers in Neurorobotics, vol. 16, Sep. 2022.
- K. Scheck and T. Schultz, “Multi-speaker speech synthesis from electromyographic signals by soft speech unit prediction,” in ICASSP, 2023.
- A. Kapur, S. Kapur, and P. Maes, “Alterego: A personalized wearable silent speech interface,” in 23rd International conference on intelligent user interfaces, 2018.
- M. Vaiman and E. Eviatar, “Surface electromyography as a screening method for evaluation of dysphagia and odynophagia,” Head & face medicine, vol. 5, no. 1, pp. 1–11, 2009.
- V. Gupta, N. P. Reddy, and E. P. Canilang, “Surface emg measurements at the throat during dry and wet swallowing,” Dysphagia, 1996.
- C. M. Sommerich, S. M. Joines, V. Hermans, and S. D. Moon, “Use of surface electromyography to estimate neck muscle activity,” Journal of Electromyography and Kinesiology, 2000.
- F. Lecluse, M. Brocaar, and J. Verschuure, “The electroglottography and its relation to glottal activity,” Folia Phoniatrica et Logopaedica, vol. 27, no. 3, pp. 215–224, 1975.
- P. Chen, L. Chen, and X. Mao, “Content classification with electroglottograph,” in Journal of Physics: Conference Series, vol. 1544, no. 1. IOP Publishing, 2020, p. 012191.
- P. Laferriere, E. D. Lemaire, and A. D. C. Chan, “Surface electromyographic signals using dry electrodes,” IEEE Transactions on Instrumentation and Measurement, vol. 60, no. 10, p. 3259–3268, Oct. 2011.
- A. C. Myers, H. Huang, and Y. Zhu, “Wearable silver nanowire dry electrodes for electrophysiological sensing,” RSC Advances, vol. 5, no. 15, p. 11627–11632, 2015.
- Y. M. Chi, T. P. Jung, and G. Cauwenberghs, “Dry-contact and noncontact biopotential electrodes: Methodological review,” IEEE Reviews in Biomedical Engineering, vol. 3, pp. 106–119, 2010.
- R. Kaveh, N. Tetreault, K. Gopalan, J. Maravilla, M. Lustig, R. Muller, and A. C. Arias, “Rapid and scalable fabrication of low impedance, 3d dry electrodes for physiological sensing,” Advanced Materials Technologies, p. 2200342, 5 2022.
- S. Stjerna, P. Alatalo, J. Mäki, and S. Vanhatalo, “Evaluation of an easy, standardized and clinically practical method (sureprep) for the preparation of electrode–skin contact in neurophysiological recordings,” Physiological Measurement, 2010.
- J. Kornej, H. Lin, L. Trinquart, C. R. Jackson, D. Ko, E. J. Benjamin, and S. R. Preis, “Neck circumference and risk of incident atrial fibrillation in the framingham heart study,” Journal of the American Heart Association, vol. 11, no. 4, p. e022340, 2022.
- A. Zhou, S. R. Santacruz, B. C. Johnson, G. Alexandrov, A. Moin, F. L. Burghardt, J. M. Rabaey, J. M. Carmena, and R. Muller, “A wireless and artefact-free 128-channel neuromodulation device for closed-loop stimulation and recording in non-human primates,” Nature Biomed. Eng., vol. 3, no. January, pp. 15 – 26, 2019.
- R. Kaveh, J. Doong, A. Zhou, C. Schwendeman, K. Gopalan, F. L. Burghardt, A. C. Arias, M. M. Maharbiz, and R. Muller, “Wireless user-generic ear eeg,” IEEE Transactions on Biomedical Circuits and Systems, vol. 14, pp. 727–737, 8 2020.
- A. Moin, A. Zhou, A. Rahimi, A. Menon, S. Benatti, G. Alexandrov, S. Tamakloe, J. Ting, N. Yamamoto, Y. Khan, F. Burghardt, L. Benini, A. C. Arias, and J. M. Rabaey, “A wearable biosensing system with in-sensor adaptive machine learning for hand gesture recognition,” Nature Electronics, vol. 4, pp. 54–63, 2021.
- B. C. Johnson, S. Gambini, I. Izyumin, A. Moin, A. Zhou, G. Alexandrov, S. R. Santacruz, J. M. Rabaey, J. M. Carmena, and R. Muller, “An implantable 700μ𝜇\muitalic_μW 64-channel neuromodulation IC for simultaneous recording and stimulation with rapid artifact recovery,” IEEE Symp. VLSI Circuits, pp. C48–C49, 2017.
- F. Eyben, M. Wöllmer, and B. Schuller, “Opensmile: the munich versatile and fast open-source audio feature extractor,” in Proceedings of the 18th ACM international conference on Multimedia, 2010.
- S. Chen, C. Wang, Z. Chen, Y. Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao, J. Wu, L. Zhou, S. Ren, Y. Qian, Y. Qian, M. Zeng, and F. Wei, “Wavlm: Large-scale self-supervised pre-training for full stack speech processing,” JSTSP, 2021.
- S. wen Yang et al., “SUPERB: Speech Processing Universal PERformance Benchmark,” in Interspeech, 2021.
- C. J. Cho et al., “Evidence of vocal tract articulation in self-supervised learning of speech,” in ICASSP, 2023.
- P. Wu et al., “Speaker-independent acoustic-to-articulatory speech inversion,” in ICASSP, 2023.
- A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems, 2017.
- W.-N. Hsu, B. Bolte, Y.-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,” TASLP, 2021.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.