QAPruner: Quantization-Aware Vision Token Pruning for Multimodal Large Language Models

Abstract: Multimodal LLMs (MLLMs) have shown strong reasoning ability, but their high computational and memory costs hinder deployment in resource-constrained settings. While Post-Training Quantization (PTQ) and vision token pruning are standard compression techniques, they are usually treated as independent optimizations. In this paper, we show that these two techniques are strongly coupled: naively applying semantic-based token pruning to PTQ-optimized MLLMs can discard activation outliers that are important for numerical stability and thus worsen quantization errors in low-bit regimes (\textit{e.g.}, W4A4). To address this issue, we propose a quantization-aware vision token pruning framework. Our method introduces a lightweight hybrid sensitivity metric that combines simulated group-wise quantization error with outlier intensity. By combining this metric with standard semantic relevance scores, the method retains tokens that are both semantically informative and robust to quantization. Experiments on standard LLaVA architectures show that our method consistently outperforms naive integration baselines. At an aggressive pruning ratio that retains only 12.5\% of visual tokens, our framework improves accuracy by 2.24\% over the baseline and even surpasses dense quantization without pruning. To the best of our knowledge, this is the first method that explicitly co-optimizes vision token pruning and PTQ for accurate low-bit MLLM inference.

• The paper introduces a unified framework that integrates quantization simulation with vision token pruning to maintain numerical stability in low-bit MLLMs.
• It employs a hybrid sensitivity metric that combines group-wise quantization error and global outlier intensity to rank tokens effectively.
• Experiments on LLaVA models demonstrate a 2.24% improvement in accuracy, showcasing superior performance in aggressive pruning regimes.

Quantization-Aware Vision Token Pruning in MLLMs: An Expert Overview of QAPruner

Context and Motivation

Multimodal LLMs (MLLMs) have demonstrated advanced capabilities in vision-language reasoning due to architectures that blend strong vision encoders with LLM backbones. However, their extensive computational and memory demands hinder practical deployment, especially in resource-constrained environments. Conventional optimization strategies such as Post-Training Quantization (PTQ) and vision token pruning are often adopted independently to mitigate these costs: PTQ reduces numerical precision to lower memory demands, while token pruning discards less-informative vision tokens to reduce sequence length and speed up inference.

A critical observation addressed in "QAPruner: Quantization-Aware Vision Token Pruning for Multimodal LLMs" (2604.02816) is that these two compression strategies are strongly coupled. Standard, semantic-based token pruning applied to PTQ-optimized MLLMs can inadvertently remove activation outliers crucial for numerical stability in quantized models, resulting in a marked degradation in performance in low-bit regimes (e.g., W4A4). This non-trivial interaction motivates the need for a unified approach.

Methodological Advances

QAPruner proposes a quantization-aware framework for vision token pruning, tailored specifically for PTQ-optimized MLLMs. The approach is distinguished by a hybrid token sensitivity metric that fuses simulated group-wise quantization error (capturing local information loss post quantization) with global outlier intensity (estimating magnitude of activation extremes) for each visual token. Unlike prior quantization-agnostic approaches, this sensitivity metric explicitly ranks tokens not only by semantic relevance but also by their contribution to quantization robustness.

Key technical components include:

• Group-wise Quantization Simulation: For each token, features are partitioned into channel groups, simulating the quantization process based on local dynamic ranges and quantization error, thereby reflecting actual information retention after quantization.
• Global Outlier Intensity Measurement: The token-wise maximum channel range identifies outlier-rich tokens, empirically shown to be crucial for numerical fidelity in low-bit quantization.
• Composite Token Scoring: The final pruning score is a convex combination of standard semantic scoring and the quantization sensitivity metric, with a tunable hyperparameter controlling the tradeoff.

This framework is entirely training-free and calibration-based, fitting seamlessly as a post-processing step after standard PTQ.

Empirical Results and Strong Claims

Comprehensive experiments on LLaVA-1.3 and 1.5 (7B/13B) models and the ScienceQA benchmark validate QAPruner's superiority in low-bit inference settings. The method is shown to:

• Consistently outperform naive baseline approaches where token pruning and PTQ are applied independently.
• Achieve a 2.24% absolute improvement in accuracy over the baseline when retaining only 12.5% of original vision tokens, notably surpassing even dense quantized baselines in certain settings.
• Maintain higher accuracy across aggressive pruning ratios compared to semantic-only pruning, with the performance gap widening under more severe token constraints.
• Demonstrate through ablation that composite metrics (combining group-wise quantization simulation and outlier intensity) yield the most robust outcomes, outperforming variant sensitivity scores such as $L_1$, $L_2$, or variance-based methods.

These empirical findings highlight a nontrivial, and in some cases counterintuitive, implication: tokens less relevant semantically may be indispensable for quantization stability, and discarding them undermines downstream performance more than previously recognized.

Theoretical and Practical Implications

The QAPruner approach moves beyond the orthogonality assumption prevalent in the compression literature for MLLMs, establishing the necessity of joint consideration for quantization and token selection. The explicit integration of numerical stability criteria into the pruning process opens new research directions for efficient multimodal inference, including:

• Deployment on edge devices: The significant reduction in memory and computation without accuracy loss broadens the applicability of MLLMs in mobile or real-time contexts.
• Transferability to broader model classes: The methodology generalizes to any PTQ-ready vision-language architecture using vision token representations.
• Influence on future pruning algorithms: This work calls for token selection metrics that are sensitive to downstream quantization characteristics, not just semantic retention.

Outlook and Future Developments

QAPruner represents a methodological shift by highlighting the intertwined nature of quantization and token pruning in real-world MLLM deployment. Potential future developments include extending this paradigm to more complex quantization schemes (e.g., adaptive or mixed-precision quantization), integration with end-to-end training objectives for joint compression optimization, and generalization to other modalities where feature pruning is relevant.

The broader implication is an emerging principle: compression methods for large models must be designed in concert rather than as independent modules, particularly as quantized inference becomes ubiquitous.

Conclusion

"QAPruner: Quantization-Aware Vision Token Pruning for Multimodal LLMs" (2604.02816) provides both analytical insight and a concrete solution to the challenges of efficient MLLM inference under quantization. By coupling hybrid sensitivity metrics with semantic relevance for vision token selection, QAPruner delivers measurable accuracy gains in aggressive pruning scenarios and establishes new best practices for post-training model compression in multimodal architectures. This work substantiates the necessity of quantization-awareness in token pruning and sets a precedent for unified methodologies in efficient multimodal AI systems.