Mestra: Exploring Migration on Virtualized CGRAs

Abstract: As modern Coarse Grain Reconfigurable Arrays (CGRAs) grow in size, efficient utilization of the available fabric by a single application becomes increasingly difficult. Existing CGRA mappers either fail to utilize the available fabric or rely on rigid static code transformations with limited adaptability. Multi-tenant CGRAs have emerged as a promising solution to increase hardware utilization, but current attempts fail to address key challenges such as fabric fragmentation and live migration. To address this gap, we present Mestra, an end-to-end system for CGRA multi-tenancy that supports dynamic scheduling and resource allocation in a shared environment. Mestra addresses fabric fragmentation caused by kernels completing out of order by supporting both stateless and stateful live kernel migration as a de-fragmentation mechanism. We assess our solution on an Alveo-U280 data-center-grade FPGA card, reporting area, frequency, and power. Performance is evaluated using routines from the PolyBench benchmark suite and kernels derived from common machine learning operators. Results show that spatial sharing of the available fabric across multiple users improves workload makespan by up to 70.48%, while live kernel migration reduces tail latency on fragmented layouts by up to 29.60%. The custom tightly coupled controller and read-back paths required for virtualization and stateful migration introduce a LUT cost of 0.13% per region. Our evaluation reveals that multi-tenancy is important for efficient CGRA utilization, and live kernel migration can further improve performance by recovering fragmented space with minimal hardware cost.

• The paper introduces a novel live migration approach for virtualized CGRAs that mitigates spatial fragmentation in multi-tenant systems.
• It employs both stateless and stateful migration mechanisms, with stateful migration achieving up to 29.60% reduction in tail latency and 30.60% lower turnaround time.
• Hardware emulation and simulation reveal significant wait time and latency improvements, validating the viability of the proposed architecture with negligible LUT overhead.

Architectural Innovations in Multitenant Virtualized CGRAs: An Analysis of "Mestra: Exploring Migration on Virtualized CGRAs"

Introduction

"Mestra: Exploring Migration on Virtualized CGRAs" (2604.04694) introduces an end-to-end system architecture and runtime for virtualization, multitasking, and migration on CGRAs, with an explicit focus on combating spatial fragmentation through both stateless and stateful live kernel migration. The work targets major bottlenecks in dynamic multi-tenant CGRA utilization, analyzing fragmentation dynamics, quantifying performance improvements, and characterizing hardware overheads of virtualization and migration mechanisms. The authors position their approach within the context of prior art in coarse-grain architectures, partially reconfigurable FPGAs, and schedulability theory, noting the unique challenges and gaps specific to CGRA virtualization.

System Architecture and Execution Model

Mestra deploys a tiled architecture, partitioning a CGRA fabric into homogeneous, modular regions. Each region contains a cluster of load/store (LS) and function-compute (FC) PEs, interconnected in a 2D mesh, and is governed by a tightly coupled controller with hardware support for virtualization, control, and migration primitives.

Figure 1: Heterogeneous grid of PEs arranged on a mesh point to point network; modular regions provide full 2D allocation flexibility.

This organization exposes regions as virtual resource units (vCGRA regions). These can be elastically merged into larger allocations when kernel requirements exceed a single region's footprint, enabling high compute density for data-parallel kernels and flexible bin-packing for heterogeneously shaped workloads. The architecture offloads fine-grained per-region control to dedicated controllers, encapsulating scheduling, configuration, state capture, migration, and error signaling in a hardware-software co-design.

Figure 2: FSM of the tightly coupled controller, delineating legal state transitions and minimal command set for region control.

Figure 3: Per-PE type state-critical register elements exposed for migration and snapshot operations.

System integration is achieved via a host-side shell communicating over PCIe, supporting kernel offloading, configuration transport, state snapshot exchange, and runtime scheduling through a hypervisor.

Figure 4: System-level view: hypervisor dispatches user kernels, dynamically allocates vCGRA resources, and manages concurrent execution via the virtualization shell.

Multi-Tenancy, Fragmentation, and Migration Mechanisms

Mestra's virtualization strategy statically partitions the array into $k$ regions, but virtual merges and runtime migration allow dynamic reshaping in response to demand. The runtime adopts a greedy placement and compaction policy, tracking virtual resource maps to detect fragmentation—situations where the aggregate free area suffices for a kernel, but no sufficiently large contiguous region exists.

Figure 5: Tiled, multi-tenant architecture enables reduced average wait times and concurrent kernel scheduling, mitigating serialized queuing delays.

Fragmentation arises invariably with dynamic allocation and out-of-order completion typical in multiprogrammed, data-center-style workloads.

Figure 6: Illustration of fabric fragmentation: finished kernels leave noncontiguous holes; migration of an active kernel enables admission of a blocked, larger kernel by creating contiguous space.

Two migration mechanisms are architected:

• Stateless migration: Kernel is interrupted, state is not preserved; kernel restarts from zero on a new region. Lower implementation overhead, but incurs lost work/time.
• Stateful migration: Full runtime state (AGU progress, PE registers, unconsumed data, local memories) is snapshotted and restored on a new region, ensuring forward progress. More hardware complexity but essential for non-restartable or long-running workloads.

A tolerance threshold for stateless migration avoids unnecessary migration of nearly complete kernels, balancing progress loss versus fragmentation recovery.

Evaluation and Quantitative Results

Extensive evaluation is conducted via hardware emulation (on Xilinx Alveo-U280) and high-level simulation, leveraging PolyBench and ML kernels with varying footprint and execution characteristics to generate realistic, highly fragmented conditions.

Hardware Results: Under a $(4\times4)$ vCGRA grid (240 total PEs):

• Mean wait time decreased by 91.39%
• $P95$ tail latency reduced by 68.29%
• Mean turnaround time improved by 76.07%

These improvements are realized despite increased mean execution time (by $3.42\times$), attributed to memory interference, but total turnaround time reduces by up to $8.27\times$.

Figure 7: Overall hardware emulation: dramatic reductions in wait, tail latency, and turnaround time metrics with the tiled, multi-tenant architecture.

Figure 8: Comparison: wait time reduced $11.61\times$, but execution time increases under contention; total TAT improved up to $8.27\times$.

Simulation Results: Fragmentation-intensive workloads reveal:

• Stateless migration (with threshold) yields minor improvements, less than $3\%$ across metrics, due to work loss on migration.
• Stateful migration achieves up to $29.60\%$ reduction in P95 tail latency and $30.60\%$ lower mean turnaround time over the baseline.
• The number of migrations shows statistically significant but weak correlation with realized performance gain—quality of migration (i.e., its ability to enable block-removed placements) dominates over raw count.

  Figure 9: Simulation: stateless versus stateful migration under comparable fragmentation; stateful delivers consistent gains, stateless limited by progress loss.

  

  Figure 10: Spread of migrations versus performance gain; gains are not simply proportional to migration count, underscoring the importance of intelligent migration targeting.

Resource Overheads:

• Full system (shell and CGRA) synthesis achieves a cost of $(4\times4)$0 LUT usage on the U280.
• Per region overhead for virtualization and migration is 0.13% LUTs, indicating strong scalability and minimal resource amplification.

Implications and Future Directions

Mestra demonstrates that virtualization and multi-tenancy—well-established in FPGA systems—yield significant improvements in CGRA utilization only if runtime migration and de-fragmentation mechanisms are integrated. The introduction of stateful migration is particularly notable: it enables strictly correct, low-overhead forward progress across region moves, a critical property for non-restartable tasks and long-running scientific or ML workflows. Results indicate that stateful migration is indispensable for maintaining high system utilization and predictable tail latency under fragmented multiprogrammed workloads.

Practically, a negligible hardware cost is paid for this added flexibility. Theoretically, the architecture motivates future research into migration-aware compilers, predictive migration (e.g., Q-learned or profile-driven), and scheduling policies that can intelligently balance migration cost versus gain beyond the greedy heuristic currently implemented.

As spatial accelerators and large CGRA fabrics proliferate, driven by AI/ML domain requirements, architectures that enable maximal utilization, low tail latency, and fine-grained hardware resource virtualization will become increasingly relevant. The migration mechanisms of Mestra are directly applicable to emerging multi-tenant spatial accelerators in data-center and HPC contexts, and the snapshot methodology could inform checkpointing strategies in broader reconfigurable systems.

Conclusion

Mestra provides a comprehensive architecture and runtime for multi-tenant, virtualized CGRAs with live migration. It is the first to quantify and systematically evaluate the interplay of fragmentation and migration within this context, demonstrating that stateful migration substantially and consistently boosts system performance metrics in a wide variety of workloads, with negligible hardware overhead. The architectural principles and mechanisms established by Mestra are poised to drive advances in dynamic spatial resource management for next-generation reconfigurable fabrics.

---

Reference:

"Mestra: Exploring Migration on Virtualized CGRAs" (2604.04694)