Global Latency-Driven Exploration

• Global latency-driven exploration is an optimization approach that prioritizes end-to-end latency reduction by jointly considering neural architectures, hardware mappings, and scheduling constraints.
• It employs unified search methods combining analytic latency estimation and empirical measurement, using combinatorial search and pruning to navigate complex design spaces.
• Applied across neural accelerators, DNN pruning, and interactive UX systems, this framework achieves significant improvements in speed, energy efficiency, and system-level performance.

Global latency-driven exploration refers to the class of optimization frameworks, algorithms, and methodologies in which end-to-end (EtoE) latency constitutes the primary design or pruning objective. These frameworks systematically enumerate, evaluate, and select configurations—spanning neural network architectures, hardware mappings, and system-level scheduling—such that the resulting system strictly satisfies global latency objectives or budgets. Unlike local or heuristically decomposed latency mitigation, global latency-driven exploration universally accounts for cross-layer, cross-module, or device-level interactions, buffer bottlenecks, and parallelism trade-offs, and combines analytic latency estimators with empirical measurements and combinatorial search. This approach has become pivotal across neural network accelerator architectures, LLM serving, system-level hardware synthesis, edge deployment DNN pruning, and interactive data exploration, as detailed below.

1. Unified Design Space Formulations in Neural Accelerators

Latency-driven exploration in tensorized neural network accelerators is anchored in the simultaneous optimization of contraction paths, hardware partitioning, and dataflow mappings. In recent frameworks, each axis is formalized:

• Tensor-contraction paths: For a tensor train (TT) layer with $D$ cores, the set $P_l$ enumerates feasible contraction orders. Paths are pruned by multiply-accumulate (MAC) cost using depth-first search to yield Top-$K$ minimal-MAC paths.
• Hardware architecture parameters: The accelerator (typically a 2D systolic array of $M_{\text{PE}} \times N_{\text{PE}}$) may adopt:
  • Monolithic execution ($C_h = \{1 \times 1\}$): full array per layer.
  • Split-X partitioning: array divided vertically/horizontally ($C_h = \{1 \times 2, 2 \times 1\}$).
  • Sub-array assignments $c_l \in C_h$ chosen per layer.
• Dataflow mappings: GEMM operations are assigned input-stationary (IS), weight-stationary (WS), or output-stationary (OS) flows $d_l \in D_l$.

Only joint exploration of $(p_l, c_l, d_l)$ reveals optimal trade-offs in compute, memory, and bandwidth, producing configurations that minimize end-to-end latency across all layers (Zhang et al., 22 Nov 2025).

2. Global Objective Functions and Search Algorithms

The central objective is to minimize total latency across the design space. For $L$ tensorized layers, let $P = \{p_l\}_{l=1}^L$, $C = \{c_l\}_{l=1}^L$, $D = \{d_l\}_{l=1}^L$, and $h \in H$ the global hardware strategy. The optimization is:

$$\{h^*,P^*,C^*,D^*\} = \underset{ h\in\mathcal H,\; \{p_l\in P_l,\;c_l\in C_h,\;d_l\in D_l\}_{l=1}^L }{\mathrm{arg\,min} \;\sum_{l=1}^L \mathrm{Cost}\bigl(p_l,\;c_l,\;d_l\bigr) }$$

Subject to all hardware and mapping constraints. In training-aware scenarios, a weighted sum of inference and training latencies is used:

$$L_{\mathrm{total}} =\alpha\sum_{l=1}^L L_{\mathrm{infer}}(p_l,c_l,d_l) +\beta\sum_{l=1}^L L_{\mathrm{train}}(p_l,c_l,d_l)$$

Search algorithms typically employ layer-wise pruning followed by hierarchical enumeration:

• For each layer, Top-$K$ MAC paths $(P_l)$ are determined.
• All combinations $(p, c, d)$ are simulated and stored.
• For each global hardware partitioning $h$, per-layer minima are summed, and the configuration $(h^*, P^*, C^*, D^*)$ yielding least total cost is selected.
• No stochastic search is required; every configuration is measured exactly once, with pruning for path equivalence and excessive cost (Zhang et al., 22 Nov 2025).

3. Latency Estimation and Measurement Models

Analytic and simulation-based models are merged to predict latency:

• Tile-compute cycles: For tiling a GEMM $(M \times K \times N)$ with tile sizes $(T_M, T_N, T_K)$, the pipeline penalty and sum of cycles over tiles are computed as:

$$C_{\mathrm{compute}} = \sum_{i=1}^{N_M}\sum_{j=1}^{N_N}\sum_{k=1}^{N_K} (T_M + T_N - 1)$$

• DRAM transfer:

$$C_{\mathrm{DRAM}} = \frac{(MK + KN + MN) \cdot \mathrm{data\_size}}{\mathrm{BW}_{\mathrm{DRAM}}}$$

• Aggregate latency:

$$L(p,c,d) = \frac{C_{\mathrm{compute}}}{f_{\mathrm{clk}}} + \max\left(0, \frac{C_{\mathrm{DRAM}} - C_{\mathrm{compute}}}{f_{\mathrm{clk}}}\right)$$

Empirical hardware measurement supersedes regression-based proxies for pruning, as seen in Archtree, where each candidate network is tested directly, providing sub-1% error bounds and tighter fit to latency budgets (Reboul et al., 2023).

4. Applications in Pruning, Synthesis, and LLM Serving

A. Structured Pruning Under Latency Budgets

Global latency-driven exploration in DNN pruning replaces offline latency proxies with on-the-fly measurement. In Archtree:

• The search is a beam-structured tree, exploring multiple sub-models in parallel.
• Latency budgets propagate as stepwise interpolants, ensuring every candidate $\hat{L}(C') \leq B$ at each level.
• Pruning steps are guided by importance scores and adaptive stride, minimizing costly measurements.
• The method preserves model accuracy while fitting latency constraints significantly tighter than alternate approaches (Reboul et al., 2023).

B. High-Level Synthesis with EtoE Latency Constraints

System-level DSE frameworks (EtoE-DSE) adapt global latency-driven exploration to embedded pipelines:

• Each block is modeled as a periodic state machine with compute (MCC), handshake, and FSM states.
• A pathfinding algorithm discovers all valid EtoE routes between specified endpoints.
• Latency is aggregated as $$L_{\mathrm{path}} = \sum_{\mathrm{FSM}} L_{\mathrm{FSM}} + \sum_{\mathrm{MCC}} L_{\mathrm{MCC}} + \sum_{\mathrm{HS}} L_{\mathrm{handshake}}$$.
• The design space is segmented by frequency assignments; Pareto-elite genetic algorithms optimize energy and area under strict latency constraints.
• The approach achieves up to 89.26% Pareto-front improvement over previous GA/SA baselines (Liao et al., 2024).

C. LLM Serving and Hardware Design Exploration

ADOR exemplifies global exploration for composite serving pipelines:

• Heterogeneous dataflow architectures combine systolic arrays (for throughput) and MAC-trees (for minimal latency).
• Cost models guide block allocations to meet strict tail-latency and throughput SLAs.
• Multi-objective optimization and Pareto filtering select architectures outperforming baseline GPUs by factors exceeding 2× in throughput and 4× in area efficiency.
• The search adapts to multi-device and batch-length scaling, maintaining high hardware utilization (Kim et al., 6 Mar 2025).

5. Latency Mitigation in Interactive Data Exploration

UX-centric global latency-driven exploration shifts mitigation to the front end. Interaction snapshots transform each user request/response into a navigable artifact:

• Each interaction spawns a “snapshot” thumbnail (loading→loaded as result arrives).
• Users interact concurrently; requests are issued in parallel and indexed by unique IDs.
• Snapshots are asynchronously filled and retrievable; history playback enables browsing past states.
• No cancellation or blocking occurs; concurrent query issuance yields reduced overall completion time without loss of accuracy, even under 5–7 second delays (Wu et al., 2018).

6. Empirical Outcomes and Comparative Analysis

Latency-driven exploration consistently yields superior results:

Context	Approach	Key Metric(s)	Improvement
Neural Accelerators	Joint DSE (Zhang et al., 22 Nov 2025)	Inference, Training Latency	Up to 4×/3.85× speedup vs. baseline
LLM Serving	ADOR (Kim et al., 6 Mar 2025)	TTFT, Area Efficiency	2.51×/4.01× over NVIDIA A100
Embedded Synthesis	EtoE-DSE (Liao et al., 2024)	Quality-of-Results (AEDRS)	Up to 89.26% better Pareto front
DNN Pruning	Archtree (Reboul et al., 2023)	Accuracy Under Budget	~6–7% higher accuracy at tight budget
Interactive UX	Snapshots (Wu et al., 2018)	Completion Time	~2× faster at 5s latency, no acc loss

Empirical results verify that accurate measurement, exact enumeration, and unified search spaces are crucial for unlocking performance and energy savings while strictly fitting latency requirements.

7. Open Directions and Considerations

Current frameworks have demonstrated global latency-driven exploration across algorithmic, architectural, and UX-driven domains. Potential directions include:

• Extension to tree-structured or branched provenance in interactive tools, supporting exploration divergence and convergence.
• Automated clustering and summarization of large snapshot histories in interactive exploration.
• Hybrid frameworks combining analytic estimators and empirical measurement, especially for emerging accelerator architectures and heterogeneous serving environments.
• Quantitative studies linking latency mitigation to higher-level insight generation and sense-making, particularly in data-intensive domains.

A plausible implication is continued integration of global latency-driven optimization mechanisms throughout both hardware and application-level stacks. As latency requirements become more stringent and multi-modal, such frameworks will remain foundational for future edge, cloud, and interactive systems.