Adaptive Local Preference Optimization

• Adaptive Local Preference Optimization (ALPO) is a framework that optimizes large language models using localized preference signals to achieve expressive and context-specific subtitle translations.
• It employs a two-stage process—initial supervised fine-tuning followed by local preference optimization—to capture subtle stylistic and content nuances at the segment level.
• Empirical results on the Multidirectional Subtitle Parallel Corpus demonstrate ALPO's effectiveness in enhancing translation fidelity, vividness, and stylistic alignment across diverse genres.

Adaptive Local Preference Optimization (ALPO) is a preference alignment framework designed for fine-grained instruction tuning of LLMs, with a particular focus on expressive and vivid machine translation of subtitles in visual media. It addresses the need for domain customization and fine control over translation style, moving beyond generic translation objectives to explicitly optimize for local, context-specific user or curator preferences. ALPO was proposed and empirically validated in the context of the Multidirectional Subtitle Parallel Corpus (MuSC), which provides large-scale, high-quality bilingual data for six language pairs spanning diverse genres, periods, and translation norms (Cui et al., 1 Feb 2026).

1. Motivation and Context

ALPO emerges from a recognition of two trends in LLM development: (1) the widespread adoption of reinforcement learning from human feedback (RLHF) and preference-based methods for instruction alignment, and (2) growing demand for domain-sensitive translation, especially in subtitling where literal, liberal, expressive, and contextually vivid translations are required depending on user or platform expectations.

Traditional LLM tuning techniques have generally relied on a global preference signal—such as ranking model outputs according to general user preference or BLEU-style metrics—often leading to oversmoothing and suppression of fine stylistic nuance. ALPO explicitly targets this gap by incorporating local preference signals at the utterance or segment level, aligning the model’s outputs closely with granular, segment-specific quality judgments arising in the translation of dialogue-rich media (Cui et al., 1 Feb 2026).

2. Multidirectional Subtitle Parallel Corpus (MuSC)

ALPO was developed and evaluated using MuSC, a corpus purpose-built for preference-based translation research. MuSC comprises over 8 million bilingual subtitle line-pairs across six language directions (English→German, English→French, English→Chinese, Korean→Chinese, Chinese→English, Chinese→Thai) sampled from the Youku platform (Cui et al., 1 Feb 2026). Key characteristics include:

• Coverage of four program types: films, TV series, documentaries, animation.
• Balanced representation from both same-language-family and cross-family translation pairs.
• Rigorous alignment via timestamp-based heuristics, restricting aligned pairs to those with start time difference $\leq$ 0.7 seconds.
• Automatic filtering for outlier line length and removal of non-dialogue cues.
• 10% program-level hold-out for testing per direction, with remaining data split for supervised fine-tuning (SFT) and preference alignment.

MuSC is thus both the backbone of ALPO’s training and a testbed for robust, local-quality-sensitive translation modeling.

3. Methodological Framework of ALPO

Adaptive Local Preference Optimization is instantiated as a two-stage process:

Stage 1: Supervised Fine-Tuning (SFT)

LLMs are initially trained on aligned subtitle line pairs. This stage establishes general translation competency.

Stage 2: Local Preference Optimization

A preference dataset is constructed by collecting multiple candidate translations per source utterance and having LLM-based or human judges rank or score these alternatives according to segment-level translation quality. Segment-level evaluation focuses on attributes such as faithfulness, expressiveness, vividness, and adherence to stylistic conventions.

The ALPO algorithm then finetunes the LLM by optimizing a ranking loss or reward signal that reflects these local judgments. Unlike global preference alignment, which aggregates signals over entire documents or datasets, ALPO ensures that learning is driven by contextually relevant, construct-specific feedback at the granularity of subtitle lines or utterances.

A salient feature of this design is the reliance on LLMs as both generators and evaluators (judges) of translation quality, found to be reliable in distinguishing literal, liberal, and expressive translation options in this domain (Cui et al., 1 Feb 2026).

4. Quality Control and Data Alignment

Accurate segment-level preference optimization depends critically on high-quality alignment of translation pairs. In MuSC, line-pair alignment is achieved by windowed matching of source and target lines within a start-time difference of 0.7 seconds, discarding lines with no suitable temporal match. Extremely long or short lines (outside 1–40 tokens) are excluded and merged/split lines, common in human-supplied subtitles, are handled by the timestamp approach, which recovers 1:1 mappings without manual intervention (Cui et al., 1 Feb 2026).

This procedure preserves the correspondence between spoken utterances and their translations, allowing local preference signals to be unambiguously associated with specific units of meaning and context.

5. Empirical Results and Efficacy

ALPO demonstrates strong multidimensional performance gains in translation quality. Experimental results, as reported in the original work, indicate "outstanding performance in multidimensional evaluation of translation quality," with direct improvements observable in segment-level fidelity, vividness, and stylistic alignment, as judged by both LLM and human evaluators (Cui et al., 1 Feb 2026).

The preference optimization approach readily supports different translation norms—literal, liberal, or expressive—by altering reward models or judge preferences, substantiating ALPO as a flexible tool for domain-adaptive translation constraints.

6. Applications and Implications

ALPO is designed for the alignment of LLM outputs to fine-grained user or institutional preferences, with immediate application in subtitle translation. The framework is particularly suited to tasks requiring nuanced, segment-local adaptation—such as adapting translations for genre, demography, or production house style guides—which are not adequately captured by corpus-level BLEU or similar aggregate metrics.

A plausible implication is that ALPO-style optimization could generalize beyond translation to any domain where local preferences vary substantially by context, such as style transfer, summarization, or dialogue generation.

7. Limitations and Future Prospects

ALPO’s dependence on reliable local preference data is both a strength and a constraint. Large-scale acquisition of high-quality, localized preference judgments remains labor-intensive. The methodology also presupposes robust alignment at the utterance or segment level, a requirement satisfied in MuSC but less tenable in noisier domains.

The authors identify potential for further research in expanding ALPO to less structured or less temporally-aligned domains, integrating human-in-the-loop validation, and refining reward models for cross-linguistic and cross-domain generality (Cui et al., 1 Feb 2026). The approach’s reliance on LLMs as both reward models and evaluators also underscores the importance of continuing advances in LLM evaluation fidelity.