Explicit Learning and the LLM in Machine Translation
Abstract: This study explores an LLM's ability to learn new languages using explanations found in a grammar book$\unicode{x2014}$a process we term "explicit learning." To rigorously assess this ability, we design controlled translation experiments between English and constructed languages generated$\unicode{x2014}$by specific cryptographic means$\unicode{x2014}$out of Latin or French. Contrary to previous studies, our results demonstrate that LLMs do possess a measurable capacity for explicit learning. This ability, however, diminishes as the complexity of the linguistic phenomena to be learned increases. Supervised fine-tuning on ad hoc chains of thought significantly enhances LLM performance but struggles to generalize to typologically novel or more complex linguistic features. These findings point to the need for more diverse training sets and alternative fine-tuning strategies to further improve explicit learning by LLMs, benefiting low-resource languages typically described in grammar books but lacking extensive corpora.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Explain it Like I'm 14
Overview
This paper studies whether LLMs—the AI systems that write and translate text—can learn new grammar rules on purpose by reading explanations, like a student using a grammar book. The authors build “secret-code” versions of real languages and use them to test if an LLM can read instructions about the grammar and then translate correctly, even when it has never seen that language before.
What questions does the paper ask?
The paper focuses on three simple questions:
- Can an LLM do explicit learning, meaning: read grammar explanations and use them to translate?
- How does this ability change when grammar gets harder (for example, more rules at once or more complex word endings)?
- Does training the LLM with step-by-step reasoning (“showing its work”) help it learn and apply grammar rules better? And does that learning carry over to new kinds of grammar?
How did they test this?
Think of it like teaching a friend a new language, but with a twist:
- Secret-code languages: The authors took real languages (French and Latin) and turned them into “constructed languages” (conlangs) by encrypting them—like using a strong cipher so the LLM can’t recognize familiar words and cheat. This way, the LLM must rely on the provided grammar instructions and dictionary, not its memory.
- Grammar books and dictionaries: They prepared short, clean explanations of grammar (like how to make plurals, where adjectives go, how verbs change) and small dictionaries of words. Some words appear in examples, some only in explanations, and some not at all—so the model sometimes has to apply rules without seeing an example.
- Translation tasks: The LLM had to:
- Translate English sentences into the secret-code language.
- Translate from the secret-code language back into English.
- These tasks test whether the model can apply rules to create or understand the right forms.
- Different information setups:
- W: only a dictionary (minimal help).
- W+IB: dictionary plus example sentence pairs accidentally present in the grammar text (some help from copying).
- W+G: dictionary plus full grammar explanations (the “explicit learning” condition).
- “Show your work” training: They fine-tuned the LLM using chain-of-thought (CoT), which means training it to write out its reasoning steps, like showing how it applies grammar rules before giving the final translation. This is like teaching a student to solve problems step by step, not just give answers.
What did they find, and why does it matter?
Here are the main results, explained simply:
- LLMs can explicitly learn from grammar explanations. Even the base model (without extra training) improved when given grammar rules rather than just a dictionary. This proves the model isn’t only guessing—it can read an explanation and use it.
- Complexity makes it harder. When tasks combined multiple rules (for example, verb endings plus plural nouns plus adjective agreement), performance dropped sharply. Especially when converting masculine forms to feminine plurals in French-like grammar, the base model struggled.
- “Show your work” helps a lot. After fine-tuning on chain-of-thought examples, the model got much better at using grammar explanations to translate—especially for the French-like conlangs it was trained on. In many cases, it went from very low accuracy to strong, reliable performance.
- But generalization is limited. When the fine-tuned model faced Latin-like secret languages that it hadn’t been trained on, improvements didn’t carry over. In other words, learning how to handle one type of grammar didn’t automatically mean it could handle very different grammar systems.
Why this matters:
- It shows that LLMs can be taught with explicit instructions, not just by feeding them lots of examples. That’s promising for languages that don’t have many data sources.
- It also warns that models may need targeted training for different grammar types—what works for one language doesn’t always transfer to another.
What could this mean for the future?
- Better help for low-resource and endangered languages: If an LLM can read a linguist’s grammar notes and apply them, it could help translate or process languages with very little data, supporting preservation and access.
- New training strategies: Fine-tuning on step-by-step reasoning boosts learning, but to make that learning generalize, models likely need:
- More diverse training sets covering many grammar types.
- Smarter fine-tuning methods that teach deeper, reusable patterns.
- Easier collaboration: Tools that understand metalinguistic explanations (the “language-about-language” that grammars use) could help linguists and AI researchers work together more effectively.
In short: LLMs can learn grammar rules on purpose from explanations and use them to translate. They do better when trained to reason step by step, but they still struggle with new or more complex grammar. Improving diversity in training and refining fine-tuning methods could make them more flexible learners, helping real-world translation—especially where data is scarce.
Collections
Sign up for free to add this paper to one or more collections.