Global Compression Commander: Plug-and-Play Inference Acceleration for High-Resolution Large Vision-Language Models

Abstract: Large vision-LLMs (LVLMs) excel at visual understanding and reasoning, but face efficiency challenges due to quadratic complexity in processing long multimodal contexts. While token compression techniques can reduce computational costs, existing approaches are designed for single-view LVLMs and fail to consider the unique multi-view characteristics of recent high-resolution LVLMs with dynamic tiling. While existing methods treat all tokens uniformly, our analysis reveals that global thumbnails can naturally guide the compression of local crops by providing holistic context for informativeness evaluation. In this paper, we first analyze dynamic tiling strategy comprehensively, revealing both the complementary nature between thumbnails and crops, and the distinctive characteristics across different crops. Based on our observations, we propose "Global Compression Commander" (i.e., GlobalCom$^2$), a novel plug-and-play token compression framework for HR-LVLMs. GlobalCom$^2$ leverages thumbnail as the "commander" to guide the compression process of local crops, adaptively preserving informative details while eliminating redundancy. Extensive experiments show that GlobalCom$^2$ maintains over 90\% performance while compressing 90\% visual tokens, reducing FLOPs and peak memory to 9.1\% and 60\% respectively across multiple benchmarks. Our code is available at https://github.com/xuyang-liu16/GlobalCom2.

• The paper introduces GlobalCom², a training-free token compression method that retains over 90% accuracy with just 10% of tokens.
• The technique uses a global-to-local attention strategy where thumbnails guide the retention of semantically important tokens.
• Experiments across ten benchmarks demonstrate significant speedups, highlighting its potential for resource-constrained multimodal applications.

Compression with Global Guidance: Towards Training-free High-Resolution MLLMs Acceleration

The paper "Compression with Global Guidance: Towards Training-free High-Resolution MLLMs Acceleration" presents a novel approach, termed GlobalCom$^2$, designed to address the computational inefficiencies associated with Multimodal LLMs (MLLMs) when processing high-resolution visual data. The increasing demand for models like LLaVA-NeXT that can handle complex vision-language tasks has highlighted the quadratic complexity problem caused by the extended sequence length of visual token inputs. This research provides a significant contribution to the field of AI by proposing a method that accelerates the inference of MLLMs without the need for retraining or significant architectural changes.

The core contribution of this work is the development of GlobalCom$^2$, a training-free token compression technique. This approach is specifically designed to optimize the processing of high-resolution images in MLLMs by effectively compressing visual tokens. GlobalCom$^2$ achieves this by employing a "global-to-local" guidance strategy, where global thumbnail information is utilized to direct the compression process for individual image crops. This ensures that semantically redundant tokens are pruned while critical local details are conserved, enhancing the model's efficiency and performance.

The methodology is divided into two key stages. Firstly, the thumbnail image acts as a global commander, assessing the importance of each crop and determining the retention ratios for token preservation. This allocation is computed using the scaled attention values between tokens and the [CLS] token from the visual encoder's attention maps, ensuring that more informative segments maintain a higher proportion of tokens. Secondly, the method evaluates token importance within each crop by considering both local relevance and global significance, preserving those critical for accurate multimodal understanding.

Experimentally, the authors validate the efficacy of GlobalCom$^2$ across ten benchmarks, demonstrating its superior performance in maintaining accuracy while significantly reducing token quantity. Notably, with a retention of only 10% of the original visual tokens, GlobalCom$^2$ achieves over 90% of the original accuracy of the LLaVA-NeXT models, outperforming state-of-the-art training-free token compression methods. These results suggest that the proposed approach is highly effective in not only retaining essential visual information but also substantially speeding up the inference process.

From a practical standpoint, this work has substantial implications for deploying MLLMs in resource-constrained environments, where computational efficiency and memory utilization are critical. Theoretically, the approach underscores the importance of considering both global and local visual contexts in model architecture, which could influence future model designs and token handling strategies in multimodal machine learning.

Future research could explore the extension of GlobalCom$^2$ into more diverse modalities and tasks, potentially broadening its applicability in real-world scenarios. Furthermore, integrating this method with other efficiency-boosting techniques like quantization or knowledge distillation could yield even more substantial improvements in performance and resource utilization. As the demand for real-time, multimodal processing grows, innovations like GlobalCom$^2$ will undoubtedly play a crucial role in shaping the future landscape of AI technologies.