OmniFocus: Query-Guided Modality-Balanced Token Compression for Omni-Modal Large Language Models

Abstract: Omni modal LLMs (OmniLLMs) have attracted wide attention for their ability to jointly process audio and video, but they generate large token sequences under audio-visual inputs, leading to substantial inference cost. Existing audio-visual token compression methods often rely on unimodal guidance, overlooking the temporal locality of query-relevant evidence in audio-visual inputs and implicitly assuming that the two modalities share a temporally aligned information density distribution. We propose \textbf{OmniFocus}, a training-free query-guided token compression method for OmniLLMs that performs independent importance estimation for video and audio, enabling a modality-symmetric compression design that preserves modality-specific salient evidence while maintaining audio-visual alignment, thereby mitigating the modality bias issue that can arise from unimodal-guided compression. Experiments on the Qwen2.5-Omni model family across four audio-visual benchmarks show that OmniFocus maintains strong compressed performance at low token retention ratios and outperforms existing baselines on several major benchmark scores at 25\% token retention. On DailyOmni with Qwen2.5-Omni-7B at 25\% token retention, OmniFocus maintains 59.40 accuracy while delivering up to 1.38$\times$ prefill speedup relative to the full-token baseline, highlighting a favorable practical accuracy-efficiency trade-off.

• The paper introduces a training-free, query-guided token compression framework for omni-modal LLMs, achieving robust cross-modal alignment.
• It employs adaptive chunk-wise drop ratio allocation and dual-score token selection to optimize token retention based on query relevance.
• At 25% token retention, it maintains high accuracy and improves GPU efficiency, advancing scalable and efficient multimodal reasoning.

Query-Guided Modality-Balanced Token Compression for Omni-Modal LLMs

Overview and Motivation

Omni-modal LLMs (OmniLLMs) have demonstrated compelling capabilities for unified audio-visual reasoning across domains such as everyday perception, video QA, and empathetic understanding. Yet, the token explosion—especially under long-form audio-visual inputs—poses fundamental inference and memory bottlenecks, limiting practical deployment. Previous audio-visual token compression strategies, particularly unimodal-guided methods, either prioritize audio or visual events based on a single modality attention signal, leading to modality bias and poor cross-modal alignment (Figure 1). These approaches fail to address the temporal locality and modality-specific density of query-relevant evidence, often degrading performance when both modalities are jointly required.

Figure 1: Unimodal-guided compression preserves audio-centric performance but degrades visual/joint-task accuracy, motivating modality-balanced query-guided compression.

Methodology: OmniFocus Architecture

OmniFocus introduces a training-free, query-aware, modality-symmetric token compression framework that operates before the LLM backbone, requiring no parameter updates. The system is composed of three sequential modules (Figure 2):

1. Query-Aware Chunk Importance Scoring: Given a textual query, audio and video tokens are mapped into temporal chunks. Separate max-similarity scores (cosine in input space) quantify the query relevance for each modality and chunk, capturing localized evidence via sparse activations.
2. Adaptive Chunk-Wise Drop Ratio Allocation: Relevance scores are transformed (via softmax) into local drop ratios for each modality, calibrated under global modality-specific budgets through binary search, with standard local bounds $[0.35, 0.75]$. This allocation preserves the token budget constraint while concentrating retention on query-relevant regions.
3. Dual-Score Token Selection: For each chunk and modality, tokens are selected by balancing inter-modal association (cross-modal cosine similarity to mean features in the paired modality) and intra-modal peak scores (distinctiveness within modality). An equal split preserves both cross-modal correspondence and modality-internal evidence. Tokens are masked out pre-LLM, maximizing inference efficiency.

   Figure 2: OmniFocus workflow, with query-guided modality-specific chunk relevance, adaptive drop ratios, and dual-score token retention.

Experimental Evaluation and Compression Performance

OmniFocus is validated on Qwen2.5-Omni-3B and 7B across DailyOmni, WorldSense, OmniVideoBench, and VideoMME. Baselines include unimodal-guided OmniZip, vision-language compression DyCoke, and random sampling. Key findings include:

• Retained Performance at Extreme Compression: At 25% token retention, OmniFocus maintains 59.40 average accuracy (Qwen2.5-Omni-7B) on DailyOmni, outperforming OmniZip by 1.67 points and DyCoke by 5.03 points. Compressed performance remains robust across WorldSense and VideoMME, with best or tied-best benchmark scores achieved.
• Superior Alignment and Modality Balance: Gains are most pronounced on AV Event Alignment (up to +5.04 points over OmniZip at 7B), supporting the claim that modality-symmetric compression preserves cross-modal cues crucial for joint reasoning tasks.
• Efficiency Gains: Prefill speedup reaches 1.38x and end-to-end speedup 1.32x over full-token inference at 25% compression, with substantial GPU memory reduction (Figure 3). Performance scales with video length, delivering higher savings on longer input sequences.

  Figure 3: Relative efficiency gains (GPU memory and end-to-end time) scale with video duration, especially for strong compression.

Ablation and Analysis

Detailed ablation studies isolate the contributions of scoring, allocation, and token selection schemes:

• Chunk Relevance Scoring: Max-similarity outperforms average pooling in query-token aggregation, underlining the importance of sparse evidence localization.
• Drop Ratio Allocation: Adaptive score-based allocation yields consistent improvement over random or uniform variants.
• Inter/Intra Token Selection: Hybrid selection is optimal; intra-modal peak retention provides discriminative advantage for modalities with modality-specific evidence (Figure 4, Figure 5).
• Modality-Balanced Guidance: Joint audio-video chunk scoring surpasses single-modality guides, minimizing bias and robustly capturing the evidence distribution.

  Figure 4: OmniFocus is resilient to moderate modality-specific budget settings; aggressive compression of any modality degrades performance.

  

  Figure 5: Token selection accuracy is sensitive to inter/intra split; balanced retention is optimal across benchmarks.

Practical and Theoretical Implications

OmniFocus fundamentally shifts modality compression for OmniLLMs to a query-driven, symmetric, evidence-preserving approach that can be deployed in a training-free manner on diverse model backbones. Its efficiency improvements without retraining—crucial for long-form and real-time multimodal reasoning—enable scalability of unified perception systems in resource-limited environments. The method also reveals how attention to modality-specific query evidence and chunk alignment enhances the capacity for cross-modal reasoning, potentially extending to even more modalities (e.g., text, sensor).

Theoretically, the work undermines the premise of unimodal-guided compression as universally optimal for cross-modal LLMs, highlighting the need to combine local query relevance, adaptive token budgeting, and inter-modal association to exploit the full expressiveness of fused multimodal representations.

Future Directions

• Instance-Adaptive Budget Allocation: Moving from benchmark-level calibration to per-example adaptive retention could further optimize accuracy-efficiency trade-off, especially for queries with variable modality requirements.
• Robustness to Challenging Conditions: Extension to noisy, occluded, or temporally misaligned data would test the generality of modality-balanced compression.
• Integration with Internal LLM Attention: Augmenting compression with lightweight adaptation or explicit feedback from downstream LLM attention maps may further boost query-driven selection.

Conclusion

OmniFocus provides an efficient, training-free token compression mechanism for omni-modal LLMs, overcoming modality bias and maintaining cross-modal alignment under extreme compression. The empirical results substantiate its accuracy-efficiency gains in long-form audio-visual reasoning, with robust ablations supporting modality-symmetric design choices. The approach has direct implications for scalable deployment of unified multimodal systems and advances methodological understanding of evidence-driven cross-modal compression (2607.03050).