Adaptive Cascaded Multi-Granularity Module

• The paper's main contribution is the formulation of a cascading architecture that adaptively selects multi-granularity representations to optimize task-specific performance.
• It employs a hierarchical structure generating fine-to-coarse representations with adaptive selection techniques for vision-language, statistical, and symbolic tasks.
• Empirical results demonstrate significant gains in speed, token reduction, and interpretability across applications such as radiology report generation and multi-hop QA.

An Adaptive Cascaded Multi-Granularity Generation Module (ACMGGM) is a systems architecture for hierarchical, dynamic information processing that adaptively generates, selects, and integrates representations at multiple levels of granularity. It is implemented in diverse forms across machine learning domains, including vision-language modeling, neural-symbolic reasoning, representation learning, and report generation, with the central objective of reconciling the competing demands of efficiency, robustness, and expressivity by organizing and processing knowledge in a granularity-aware, cascading fashion.

1. Core Principles and Modular Workflow

The central idea behind ACMGGM is to construct a cascade of increasingly coarse (or, conversely, fine) representations, and then adaptively select or fuse among them according to task requirements, data statistics, and learned preferences. This principle is realized through a pipeline consisting of three characteristic stages:

• Multi-level generation of candidate representations (textual, visual, relational, or statistical) at different granularities.
• Adaptive selection, fusion, or weighting among these representations, modulated by context, instructions, or performance criteria.
• Cascaded integration or orchestration whereby processing begins from the finest (or coarsest) granularity, escalating as needed.

The structured organization of levels—typically low (local/atomic), intermediate (entity/attribute, region/sentence), and high (holistic/global)—enables cross-level feedback and dynamic adaptation, in contrast to static, single-granularity approaches (Wang et al., 2024, Xia et al., 2023, Lan et al., 2024, Wei et al., 11 Jan 2026, Wang et al., 2022).

2. Paradigmatic Implementations

Vision-LLMs

In "HPT++" (Wang et al., 2024), ACMGGM is instantiated as a prompt hierarchy spanning low (entity/attribute tokens from LLM-structured graphs), high (holistic description embeddings), and global (category-agnostic, learned vectors) levels. These prompts are injected in cascade across the layers of a Transformer text encoder; each block receives all prompt types and class tokens, facilitating cross-granularity self-attention and adaptive refinement. Structured inter-token relationships are injected via relationship-guided attention.

Visual Representation

"AVG-LLaVA" (Lan et al., 2024) applies adaptive multi-granularity selection to high-resolution visual encoding. Fine-to-coarse image token grids are generated via cascaded, non-trainable pooling. A learned Visual Granularity Router—comprising a Transformer, MLP, and aggregator—dynamically selects the granularity based on both image features and textual instructions. Only the chosen granularity is passed to the backbone LLM, optimizing both efficiency and performance.

Symbolic Reasoning and Question Answering

In the CIRAG framework (Wei et al., 11 Jan 2026), the ACMGGM is formulated as a cascading orchestration layer over a shared LLM. For each input, it attempts answer generation using the smallest available context (triples); if unsuccessful ("Unanswerable"), it escalates through sentences to full passages, terminating when sufficiency is achieved. This guarantees that answers use the minimal necessary context, balancing noise control and completeness.

Data Representations via Granular-Ball Computing

"Granular-ball computing" (Xia et al., 2023) provides a statistical instantiation of ACMGGM by iteratively covering the sample space with nested, adaptively-pure "granular-balls" (hyperspherical or hyperrectangular clusters). The process splits coarse balls only as needed, constructing a hierarchy from coarse (few, large, impure) to fine (many, small, pure) representations. Predictions and computation traverse this hierarchy, achieving reductions in data size and improved interpretability.

Multi-Granularity Fusion in Radiology Reports

AGFNet (Wang et al., 2022) utilizes a cascaded self-adaptive fusion mechanism combining global (whole-image) and local (anatomy-region) features for radiology report generation. Multi-head attention fuses global vectors with region features in a layered, summative manner, before joint encoding and final text generation.

3. Architectural Details and Mathematical Formalizations

Prompt and Token Cascades (Text/Visual)

In HPT++ (Wang et al., 2024), let $[c^{i-1}, p_g^{i-1}, p_h^{i-1}, p_l^{i-1}]$ denote the class token and multi-level prompts at Transformer block $L_i$. Outputs

$$[c^{i}, …, p_l^{i}] = L_i([c^{i-1}, p_g^{i-1}, p_h^{i-1}, p_l^{i-1}])$$

are recursively updated across layers, allowing multi-level signal exchange.

In AVG-LLaVA (Lan et al., 2024), a sequence of coarse-to-fine visual tokens $X_v^{(i)}$ is generated. Router probabilities $p = \operatorname{softmax}(Z_{\text{final}})$ select the granularity $g^*$ optimal for the given prompt and image.

Relationship-Guided Attention

Relationship matrices $M_{ij}$ are recomputed per layer, modulating self-attention weights:

$$\alpha_{ij} = \operatorname{softmax}_j\left(A_{ij} \cdot M_{ij}\right)$$

where $M_{ij} = 1+\beta$ if $(w_i, w_j)\in R$ and $1/(1+\beta)$ otherwise, enforcing structural constraints from LLM-extracted graphs (Wang et al., 2024).

Cascaded Decision Processes

In CIRAG (Wei et al., 11 Jan 2026), candidate answers $a^{(g)}$ for each granularity $g\in\{C,S,D\}$ (triples, sentences, passages) are generated, and the minimal sufficient $g^*$ is chosen as

$$g^* = \min_{\prec}\{g\in G \mid \text{Suf}(a^{(g)}) = 1\}$$

with answer $\hat a = a^{(g^*)}$.

Adaptive Splitting and Merging

Granular-ball computing (Xia et al., 2023) recursively splits balls $G=(c, r)$ into sub-balls via purity thresholds and k-means, merging where proximity and increased purity allow. The resulting multilevel structure encodes the data at several granularities, facilitating coarse-to-fine analysis and prediction.

4. Theoretical and Empirical Advantages

Efficiency and Scalability

By replacing dense or redundant fine-level representations with selected coarse "summaries," ACMGGM implementations typically reduce computational cost, token count, and memory requirements. AVG-LLaVA, for instance, achieves an 85.3% reduction in visual tokens and a 2.53× speedup on the AI2D benchmark (Lan et al., 2024). Granular-ball representations yield an order-of-magnitude speedup for SVM training, as the number of balls $m\ll n$ (Xia et al., 2023).

Precision-Robustness Tradeoff

Cascaded escalation frameworks, such as those in CIRAG or HPT++, achieve higher precision by only introducing additional (potentially noisy) context or features when necessary. Empirical studies confirm that a majority of queries terminate at lower (cheaper, less noisy) granularities, with full context required only rarely (Wei et al., 11 Jan 2026, Lan et al., 2024).

Interpretability and Structural Transparency

The multi-level organization naturally supports interpretability: granular-balls correspond to interpretable population clusters; cross-level prompt hierarchies explicate compositional semantics; and fused multi-granularity embeddings in AGFNet provide insight into the balance of holistic versus localized cues (Wang et al., 2024, Xia et al., 2023, Wang et al., 2022). Ball-based approaches yield clusters and structures that are directly visualizable and statistically characterizable.

Generalization and Robustness

ACMGGM methods have demonstrated improved domain generalization, out-of-distribution robustness, and greater resistance to noise, attributable to their smoothing and selective escalation properties (Wang et al., 2024, Xia et al., 2023). Ball-purity thresholds prevent overreaction to local label noise, while cross-level regularization imposes additional robustness in prompt-based vision-LLMs.

5. Training Paradigms and Optimization

Training approaches for ACMGGM modules differ depending on the domain:

• In HPT++ (Wang et al., 2024), both the prompt cascade and relationship-guided attention are trained end-to-end, subject to self-consistency regularization anchoring the multi-granularity encoder to a frozen textual backbone.
• AVG-LLaVA’s router is trained by aligning granularity choices with the base LMM's own answer likelihoods using a ranking loss and cross-entropy, without the need for additional human annotation (Lan et al., 2024).
• CIRAG's cascade exploits zero-shot generalization from base instruction-tuned LLMs; there is no additional supervision at the granularity selection stage (Wei et al., 11 Jan 2026).
• Granular-ball splits and merges follow data-driven heuristics or unsupervised metrics (purity, compactness), with no need for separate training (Xia et al., 2023).
• AGFNet trains its Fusion Gate and downstream decoder using captioning and detection losses in standard supervised setups (Wang et al., 2022).

6. Empirical and Application-Specific Results

The table below summarizes documented empirical outcomes in exemplary domains:

System	Efficiency Gain	Accuracy / Generalization	Interpretability
HPT++	N/A	+0.5% SOTA HM (16-shot, 11 ds)	Prompt-level, structure-aware
AVG-LLaVA	–85.3% tokens, 2.53× speedup	+1% SQA over fixed-granular	Token routing explained by router head
CIRAG	–60–70% context tokens	F1/EM > single-gran. iRAG	Cascade reveals minimum-sufficient context
Granular-ball	≥10× SVM speedup	Robust to label/adversarial	Coarse-to-fine, cluster-level visualization
AGFNet	N/A	SOTA IU-Xray/MIMIC BLEU/ROUGE	Fusion weights reveal local vs. global bias

Such results indicate that ACMGGM architectures systematically improve either computational or task efficiency, generalization, or structural transparency, depending on instantiation and use-case (Wang et al., 2024, Lan et al., 2024, Wei et al., 11 Jan 2026, Xia et al., 2023, Wang et al., 2022).

7. Limitations, Open Directions, and Extensions

Despite their versatility, ACMGGM methods are subject to certain design challenges:

• The optimal granularity structure or stopping criteria may be data- or domain-dependent and, in some instances, require automated calibration.
• Explicit fusion or escalation adds marginal latency; for some real-time settings, the overhead of router/selector modules may need further reduction (Lan et al., 2024).
• For uses in explanatory AI, the path from raw hierarchy to end-user interpretable output requires additional mapping.

Possible extensions include hybrid representations (balls, ellipsoids) (Xia et al., 2023), dynamic or data-driven thresholding, integration with fully end-to-end differentiable routers, and adaptation to temporally-evolving or streaming data.

This suggests that future ACMGGM systems may incorporate more sophisticated gating mechanisms, utilize emerging backbone architectures, or extend to broader, cross-domain knowledge hierarchies.

---

References

(Wang et al., 2024) HPT++: Hierarchically Prompting Vision-LLMs with Multi-Granularity Knowledge Generation and Improved Structure Modeling (Lan et al., 2024) AVG-LLaVA: A Large Multimodal Model with Adaptive Visual Granularity (Wei et al., 11 Jan 2026) CIRAG: Construction-Integration Retrieval and Adaptive Generation for Multi-hop Question Answering (Xia et al., 2023) Granular-ball computing: an efficient, robust, and interpretable adaptive multi-granularity representation and computation method (Wang et al., 2022) Self adaptive global-local feature enhancement for radiology report generation