Papers
Topics
Authors
Recent
Search
2000 character limit reached

A Conversational Brain-Artificial Intelligence Interface

Published 22 Feb 2024 in cs.HC, cs.AI, and eess.SP | (2402.15011v2)

Abstract: We introduce Brain-Artificial Intelligence Interfaces (BAIs) as a new class of Brain-Computer Interfaces (BCIs). Unlike conventional BCIs, which rely on intact cognitive capabilities, BAIs leverage the power of artificial intelligence to replace parts of the neuro-cognitive processing pipeline. BAIs allow users to accomplish complex tasks by providing high-level intentions, while a pre-trained AI agent determines low-level details. This approach enlarges the target audience of BCIs to individuals with cognitive impairments, a population often excluded from the benefits of conventional BCIs. We present the general concept of BAIs and illustrate the potential of this new approach with a Conversational BAI based on EEG. In particular, we show in an experiment with simulated phone conversations that the Conversational BAI enables complex communication without the need to generate language. Our work thus demonstrates, for the first time, the ability of a speech neuroprosthesis to enable fluent communication in realistic scenarios with non-invasive technologies.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (51)
  1. “Brain-computer interface technology: A review of the first international meeting” In IEEE Transactions on Rehabilitation Engineering 8.2, 2000, pp. 164–173
  2. “The thought translation device (TTD) for completely paralyzed patients” In IEEE Transactions on Rehabilitation Engineering 8.2 IEEE, 2000, pp. 190–193
  3. Thorsten O Zander and Christian Kothe “Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general” In Journal of Neural Engineering 8.2 IOP Publishing, 2011, pp. 025005
  4. Moritz Grosse-Wentrup, Donatella Mattia and Karim Oweiss “Using brain–computer interfaces to induce neural plasticity and restore function” In Journal of Neural Engineering 8.2 IOP Publishing, 2011, pp. 025004
  5. “Closing the sensorimotor loop: haptic feedback facilitates decoding of motor imagery” In Journal of Neural Engineering 8.3 IOP Publishing, 2011, pp. 036005
  6. “Estimates of the prevalence of acute stroke impairments and disability in a multiethnic population” In Stroke 32.6 Am Heart Assoc, 2001, pp. 1279–1284
  7. “Incidence and profile of inpatient stroke-induced aphasia in Ontario, Canada” In Archives of physical medicine and rehabilitation 91.2 Elsevier, 2010, pp. 196–202
  8. “Imagenet large scale visual recognition challenge” In International journal of computer vision 115.3 Springer, 2015, pp. 211–252
  9. Li Deng, Geoffrey Hinton and Brian Kingsbury “New types of deep neural network learning for speech recognition and related applications: An overview” In 2013 IEEE international conference on acoustics, speech and signal processing, 2013, pp. 8599–8603 IEEE
  10. “Human-level control through deep reinforcement learning” In Nature 518.7540 Nature Publishing Group, 2015, pp. 529–533
  11. “Human-level play in the game of Diplomacy by combining language models with strategic reasoning” In Science American Association for the Advancement of Science, 2022
  12. “Language models are few-shot learners” In Advances in neural information processing systems 33, 2020, pp. 1877–1901
  13. Willem J.M. Levelt “Speaking: From Intention to Articulation” Cambridge, Massachusetts: A Bradford Book, 1989
  14. A. Meyer, L. Wheeldon and A. Krott “Automaticity and Control in Language Processing” London: Psychology Press, 2006
  15. “A Brain-Computer Interface for High-Level Remote Control of an Autonomous, Reinforcement-Learning-Based Robotic System for Reaching and Grasping” In Proceedings of the 19th International Conference on Intelligent User Interfaces, IUI ’14 Haifa, Israel: Association for Computing Machinery, 2014, pp. 83–88
  16. “Blending of brain-machine interface and vision-guided autonomous robotics improves neuroprosthetic arm performance during grasping” In Journal of neuroengineering and rehabilitation 13.1 BioMed Central, 2016, pp. 1–12
  17. “Autonomy infused teleoperation with application to brain computer interface controlled manipulation” In Autonomous Robots 41 Springer, 2017, pp. 1401–1422
  18. “Shared control of a robotic arm using non-invasive brain–computer interface and computer vision guidance” In Robotics and Autonomous Systems 115, 2019, pp. 121–129
  19. “A brain-actuated robotic arm system using non-invasive hybrid brain–computer interface and shared control strategy” In Journal of Neural Engineering 18.4 IOP Publishing, 2021, pp. 046045
  20. Laurens R Krol and Thorsten O Zander “Towards a conceptual framework for cognitive probing” In Symbiotic Interaction: 6th International Workshop, Symbiotic 2017, Eindhoven, The Netherlands, December 18–19, 2017, Revised Selected Papers 6, 2018, pp. 74–78 Springer
  21. Laurens R Krol, Pim Haselager and Thorsten O Zander “Cognitive and affective probing: a tutorial and review of active learning for neuroadaptive technology” In Journal of neural engineering 17.1 IOP Publishing, 2020, pp. 012001
  22. “Finding a focus for quality of life with aphasia: Social and emotional health, and psychological well-being” In Aphasiology 17.4 Taylor & Francis, 2003, pp. 333–353
  23. “Attention is all you need” In Advances in neural information processing systems 30, 2017
  24. OpenAI “GPT-4 Technical Report”, 2023
  25. “Llama: Open and efficient foundation language models” In arXiv preprint arXiv:2302.13971, 2023
  26. “Dialogpt: Large-scale generative pre-training for conversational response generation” In arXiv preprint arXiv:1911.00536, 2019
  27. “Plug-and-Play Conversational Models” In Findings of the Association for Computational Linguistics: EMNLP 2020 Association for Computational Linguistics, 2020, pp. 2422–2433
  28. “A high-performance neuroprosthesis for speech decoding and avatar control” In Nature Nature Publishing Group UK London, 2023, pp. 1–10
  29. “A high-performance speech neuroprosthesis” In Nature Nature Publishing Group UK London, 2023, pp. 1–6
  30. “Speech synthesis from ECoG using densely connected 3D convolutional neural networks” In Journal of neural engineering 16.3 IOP Publishing, 2019, pp. 036019
  31. Gopala K Anumanchipalli, Josh Chartier and Edward F Chang “Speech synthesis from neural decoding of spoken sentences” In Nature 568.7753 Nature Publishing Group, 2019, pp. 493–498
  32. “Imagined speech can be decoded from low-and cross-frequency intracranial EEG features” In Nature communications 13.1 Nature Publishing Group, 2022, pp. 1–14
  33. “World’s fastest brain-computer interface: Combining EEG2Code with deep learning” In PLOS ONE 14.9 Public Library of Science, 2019, pp. 1–15
  34. Erich E. Sutter “The brain response interface: communication through visually-induced electrical brain responses” Special Issue on Computers for Handicapped People In Journal of Microcomputer Applications 15.1, 1992, pp. 31–45
  35. “A high-speed BCI based on code modulation VEP” In Journal of neural engineering 8.2 IOP Publishing, 2011, pp. 025015
  36. “Language models are unsupervised multitask learners” In OpenAI blog 1.8, 2019, pp. 9
  37. Jonathan MC Lam and Walter P Wodchis “The relationship of 60 disease diagnoses and 15 conditions to preference-based health-related quality of life in Ontario hospital-based long-term care residents” In Medical care JSTOR, 2010, pp. 380–387
  38. Alexander Wei, Nika Haghtalab and Jacob Steinhardt “Jailbroken: How does llm safety training fail?” In arXiv preprint arXiv:2307.02483, 2023
  39. “Ignore previous prompt: Attack techniques for language models” In arXiv preprint arXiv:2211.09527, 2022
  40. “Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection” In arXiv preprint arXiv:2302.12173, 2023
  41. A. Zhang “Speech Recognition (Version 3.8)” Software, 2017 URL: https://github.com/Uberi/speech_recognition
  42. “Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention” In arXiv, 2021
  43. “YAKE! Keyword extraction from single documents using multiple local features” In Information Sciences 509, 2020, pp. 257–289
  44. “Self-Supervised Contextual Keyword and Keyphrase Retrieval with Self-Labelling” In Preprints.org Preprints, 2019
  45. Maarten Grootendorst “KeyBERT: Minimal keyword extraction with BERT.” Software, 2020 URL: https://github.com/MaartenGr/KeyBERT
  46. “A comprehensive review of EEG-based brain–computer interface paradigms” In Journal of neural engineering 16.1 IOP Publishing, 2019, pp. 011001
  47. “Code-VEP vs. Eye Tracking: A Comparison Study” In Brain Sciences 8.7, 2018
  48. “Brain–computer interfaces based on code-modulated visual evoked potentials (c-VEP): a literature review” Publisher: IOP Publishing In Journal of Neural Engineering 18.6, 2021, pp. 061002
  49. “Brain–computer interfaces based on code-modulated visual evoked potentials (c-VEP): A literature review” In Journal of Neural Engineering 18.6 IOP Publishing, 2021, pp. 061002
  50. “Improving user experience of SSVEP BCI through low amplitude depth and high frequency stimuli design” In Scientific Reports 12.1 Nature Publishing Group UK London, 2022, pp. 8865
  51. “Burst c-VEP Based BCI: Optimizing Stimulus Design for Enhanced Classification with Minimal Calibration Data and Improved User Experience” In bioRxiv Cold Spring Harbor Laboratory, 2023

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 2 tweets with 20 likes about this paper.

Sign Up for Free