Adapting Large Language Models for Character-based Augmentative and Alternative Communication

Abstract: Users of Augmentative and Alternative Communication (AAC) may write letter-by-letter via an interface that uses a character LLM. However, most state-of-the-art large pretrained LLMs predict subword tokens of variable length. We investigate how to practically use such models to make accurate and efficient character predictions. We fine-tune models using a large dataset of sentences we curated in which each sentence is rated according to how useful it might be for spoken or written AAC communication. We find that using an algorithm to produce character predictions from a subword LLM provides more accurate predictions than adding a classification layer or using a byte-level model. We also find that our domain adaptation procedure is effective at improving model performance on simple, conversational text.