Unsupervised Translation of Emergent Communication

Abstract: Emergent Communication (EC) provides a unique window into the language systems that emerge autonomously when agents are trained to jointly achieve shared goals. However, it is difficult to interpret EC and evaluate its relationship with natural languages (NL). This study employs unsupervised neural machine translation (UNMT) techniques to decipher ECs formed during referential games with varying task complexities, influenced by the semantic diversity of the environment. Our findings demonstrate UNMT's potential to translate EC, illustrating that task complexity characterized by semantic diversity enhances EC translatability, while higher task complexity with constrained semantic variability exhibits pragmatic EC, which, although challenging to interpret, remains suitable for translation. This research marks the first attempt, to our knowledge, to translate EC without the aid of parallel data.

• The paper applies Unsupervised Neural Machine Translation (UNMT) to interpret emergent communication protocols developed by AI agents in referential games, addressing the lack of parallel natural language data.
• The study found that the complexity of the communication task non-monotonically affects the emergent language's properties, with moderate semantic diversity yielding the richest languages.
• Translation quality to natural language was also non-monotonic; tasks requiring discrimination between distinct concepts resulted in the most translatable emergent languages.

Unsupervised Neural Machine Translation (UNMT) offers a promising avenue for interpreting Emergent Communication (EC) protocols developed by AI agents, particularly given the inherent lack of parallel natural language (NL) corpora for novel ECs. Research explores the application of UNMT techniques to decipher ECs generated in referential games, analyzing how factors like task complexity and semantic diversity influence the characteristics of the emergent languages and their subsequent translatability into a target NL, such as English (2502.07552).

Emergent Communication Generation via Referential Games

The foundation for generating EC involves training pairs of AI agents—a Sender and a Receiver—within the framework of referential games. The Sender observes a target image and generates a discrete symbolic message intended to allow the Receiver to identify this target image from a set that includes the target and several distractor images. The agents are typically implemented using standard neural architectures: pre-trained ResNets for image feature extraction (weights often shared between Sender and Receiver for consistent visual grounding) and LSTMs for processing the sequential message data. Training employs objectives like infoNCE within frameworks such as EGG, optimizing the agents' collaborative task success (i.e., the Receiver correctly identifying the target based on the Sender's message) (2502.07552). The discrete messages successfully exchanged between communicating agent pairs during evaluation phases are collected to form monolingual EC corpora, which serve as the source language input for the subsequent UNMT phase. Communication channels are typically defined with constraints, such as a fixed vocabulary size (e.g., 64 symbols) and maximum message length (e.g., 6 symbols plus an EOS token) (2502.07552).

Unsupervised Translation Methodology

The core challenge in translating EC is the absence of paired EC-NL sentences. UNMT addresses this by leveraging large monolingual corpora in both the source (EC) and target (NL) languages. The approach detailed in (2502.07552) utilizes a UNMT system adapted from prior work (Chronopoulou et al., 2020), which involves a multi-stage process built upon a pre-trained cross-lingual model like XLM:

1. Pre-training: The model is initially pre-trained on a large monolingual corpus of the target NL (e.g., English MSCOCO image captions). This initializes the model with strong NL understanding capabilities.
2. Fine-tuning (Language Adaptation): The pre-trained model is then fine-tuned using both the monolingual EC corpus generated from the referential game and the monolingual target NL corpus. This crucial step aims to map the representations of EC symbols and NL words into a shared latent space, enabling cross-lingual transfer despite the lack of direct supervision.
3. Iterative Refinement: The translation capability is further refined through iterative cycles of back-translation and denoising autoencoding. Back-translation involves generating synthetic parallel data: the model translates NL sentences into pseudo-EC, which are then used as training targets for the EC-to-NL translation direction (and vice-versa). Denoising autoencoding tasks are applied to both corpora to improve the robustness and fluency of the LLMs in each language.

This process allows the UNMT model to learn translation mappings without any explicit EC-NL sentence pairs, relying on the shared embedding space and the statistical patterns within each monolingual corpus.

Experimental Design: Task Complexity and Semantic Diversity

To investigate how the nature of the communication task influences EC properties and translatability, experiments systematically vary the task complexity within the referential game setup (2502.07552). Complexity is operationalized by controlling the semantic diversity or similarity between the target image and the distractor images presented to the Receiver. Using the MSCOCO dataset, which offers rich image annotations (captions, categories, supercategories), different levels of complexity are instantiated:

1. Random: Distractors are chosen randomly from the entire dataset, implying low average semantic similarity to the target and making discrimination potentially easier.
2. Inter-category: Distractors belong to different object categories than the target. This forces communication to focus on distinguishing between broader concepts, potentially aligning EC structure with NL categorical distinctions.
3. Supercategory: Distractors belong to the same supercategory as the target (e.g., target is a 'giraffe', distractors are other animals like 'cow', 'zebra'). This requires finer-grained discrimination within a broad semantic group.
4. Category: Distractors are different images from the same fine-grained category as the target (e.g., different images of giraffes). This represents high contextual difficulty, potentially forcing agents to communicate subtle, instance-specific details rather than broad categorical information.

Multiple independent training runs (seeds) are performed for each complexity level to account for the stochasticity in agent training and EC emergence.

Evaluation and Results Analysis

The evaluation encompasses metrics assessing both the intrinsic properties of the generated EC and the quality of the subsequent UNMT output.

EC Properties

Standard EC metrics are employed, including:

• Task Success: Accuracy in the referential game (ACC@k).
• Linguistic Properties: Vocabulary Usage (VU), Message Entropy, Message Novelty.
• Compositionality/Structure: Topographic Similarity (TopSim), Disentanglement measures (BosDis, PosDis), Adjusted Mutual Information (AMI) comparing message clusters to ground-truth image concept clusters.

Results indicated that agents successfully learned communication protocols across all complexity levels, significantly outperforming random baselines (2502.07552). Interestingly, the relationship between task complexity and linguistic richness was non-monotonic. The hardest discrimination task (Category) resulted in EC with surprisingly low VU and Entropy, suggesting the emergence of a highly pragmatic protocol optimized for distinguishing subtle visual differences with minimal signaling effort. Conversely, the Supercategory task, requiring discrimination between semantically related but distinct concepts, yielded EC with the highest VU and Entropy, indicating a richer, more diverse vocabulary was needed (2502.07552).

UNMT Performance

Translation quality is assessed using standard MT metrics (BLEU, METEOR, ROUGE-L, BERTScore, Jaro Similarity) by comparing the UNMT output (EC translated to English) against the ground-truth MSCOCO captions associated with the original target image (reporting the maximum score across multiple reference captions). Semantic alignment is measured using CLIP Score between the translated text and the image. Lexical diversity (Type-Token Ratio) and novelty (new n-grams) of the translations are also considered.

Key findings regarding translatability include (2502.07552):

• Feasibility: UNMT successfully translated EC into coherent English, achieving scores significantly above random baselines and comparable to some low-resource UNMT benchmarks (e.g., BLEU scores ranging from ~6 to ~9).
• Impact of Task Complexity: The relationship between task complexity and translatability was also non-trivial. The Inter-category setting yielded the highest translation quality (e.g., BLEU 9.21, ROUGE-L 0.370). This suggests that tasks requiring discrimination between distinct concepts might foster EC structures more amenable to mapping onto NL concepts. The Category setting, despite producing pragmatically simpler EC (low VU/Entropy), also resulted in good translatability (e.g., BLEU 7.41, ROUGE-L 0.361), potentially because the simpler, more predictable structure was easier for UNMT to learn. The Supercategory setting, which produced the linguistically richest EC (high VU/Entropy), paradoxically resulted in the lowest translation scores (e.g., BLEU 6.08, ROUGE-L 0.343), indicating that high intra-category diversity leads to complex ECs that are challenging for UNMT to map consistently.
• Semantic Alignment: CLIP scores confirmed that translated texts were semantically relevant to the source images, though less aligned than ground-truth captions.
• Qualitative Aspects: Translations generally captured the main objects or themes but sometimes exhibited hallucinations or inaccuracies, potentially artifacts of the UNMT process or the nature of the EC itself.
• Correlation with EC Properties: Analysis suggested positive correlations between conceptual alignment in EC (AMI) and translation scores (BLEU, METEOR), while higher symbol-level disentanglement (BosDis, PosDis) correlated negatively with standard scores but positively with translation novelty. This hints that ECs encoding concepts holistically might be easier to translate directly, while highly compositional ECs encoding specific features might lead to more novel but less fluent translations.
• Protocol Uniqueness: Lack of correlation in translation performance across different random seeds for the same complexity setting highlighted that distinct EC protocols emerge even under identical conditions, each presenting unique translation challenges and opportunities.

Conclusion

The application of UNMT provides a viable, data-driven method for interpreting emergent communication protocols without requiring parallel data. The translatability of EC is shown to be intricately linked to the specifics of the generation task, particularly the nature of the discrimination required (semantic diversity). Counter-intuitively, maximum task complexity or maximum linguistic richness (entropy, vocabulary size) of the EC does not necessarily guarantee the highest translatability. Pragmatic pressures in highly specific discrimination tasks can lead to simpler, yet relatively translatable, protocols, while tasks requiring rich conceptual distinctions might yield ECs that are harder for current UNMT methods to map effectively to natural language. This line of research offers valuable tools and insights for analyzing the structure and meaning embedded within autonomously generated communication systems.