Evaluating Architectural Trade-offs in CGRAs: The Impact of Scratchpad Memory and Heterogeneity on Compute-Intensive Kernels

Abstract: Modern edge computing applications, particularly high-throughput stream processing like Vision Transformers (ViTs), demand massive spatial parallelism and efficient data movement under tight power and area constraints. Coarse-Grained Reconfigurable Architectures (CGRAs) offer a promising paradigm to balance performance, flexibility, and energy efficiency. This paper analyzes the impact of two critical CGRA design choices: processing element heterogeneity and local data reuse support. We evaluate essential computational kernels (Fast Fourier Transform (FFT) and General Matrix Multiply (GEMM)) alongside an end-to-end seizure detection transformer workload across two distinct configurations: a baseline homogeneous architecture and a heterogeneous evolution integrating specialized functional units with an Scratchpad Memory (SPM). Our evaluation demonstrates that the SPM significantly optimizes data movement, reducing memory traffic eightfold compared to a memory-less design. While the heterogeneous architecture achieves superior energy efficiency for data-shuffling tasks, the homogeneous design minimizes area overhead by 4.4x to 8.2x relative to state-of-the-art CGRAs. Furthermore, it sustains a 700 MHz operating frequency, enabling up to a 5x execution speedup over the heterogeneous configuration during matrix computations. Ultimately, this work provides an architectural roadmap for selecting CGRA fabrics based on the arithmetic intensity, performance goals, and resource envelopes of edge-scale workloads.

• The paper demonstrates that integrating scratchpad memory and heterogeneity in CGRAs significantly reduces memory overhead and accelerates compute-intensive kernels.
• It employs synthetic benchmarks and a real-world seizure detection pipeline to quantify energy, performance, and instruction savings compared to homogeneous arrays.
• The study offers design insights for optimizing CGRA architectures by balancing area, energy efficiency, and throughput based on workload characteristics.

Evaluating Architectural Trade-offs in CGRAs: The Impact of Scratchpad Memory and Heterogeneity on Compute-Intensive Kernels

Introduction

This paper investigates architectural trade-offs in Coarse-Grained Reconfigurable Arrays (CGRAs) targeting high-throughput and energy-efficient edge workloads. Specifically, it analyzes the impact of processing element (PE) heterogeneity and local scratchpad memory (SPM) integration on the efficiency of compute- and data-intensive kernels, utilizing both synthetic benchmarks (FFT, GEMM) and a real-world biomedical application (seizure detection transformer). Two platforms are systematically compared: OpenEdgeCGRA (homogeneous, memory-less) and DISCO-CGRA (heterogeneous, with SPM and specialized functional units).

Comparative Architectural Overview

The analysis isolates architectural features by leveraging the functional heritage between OE and DISCO-CGRA. OE embodies a $4 \times 4$ homogeneous mesh of identical PEs, each with local registers but no SPM, relying fully on DMA access to main memory. This design simplifies both the architecture and compilation process but suffers from extensive memory traffic during operations with poor data locality.

Figure 1: OpenEdgeCGRA architecture; a homogeneous array without local scratchpad memory, color-coded for memory, control, and compute domains.

DISCO-CGRA, as an evolution of OE, introduces two key differentiators: specialization through SIMD- and MAC-enhanced PEs, and a hierarchical memory system comprising a shared SPM and wide vector registers (VWRs). This enables effective double-buffered data reuse, reduces interconnect contention, and supports fast context switching via VLIW-style control.

Figure 2: DISCO-CGRA with heterogeneous specialized functional units and integrated scratchpad memory.

DISCO's PEs are grouped in two clusters, each with a local program counter and instruction memories. Heterogeneity is realized with dedicated load/store, multiplexer-control, and loop-control units, optimized for parallelism and efficient orchestration of multi-level tiled matrix operations.

Kernel Mapping and Dataflow Realization

The execution of a matrix multiplication kernel ($C = A \times B$) demonstrates the core differences in mapping strategies. OE maps tiles to match the $4 \times 4$ mesh, using its limited registers for both kernel metadata and operand buffering. This necessitates diagonal mapping of configuration parameters to balance register pressure, often constraining aggressive loop unrolling and unavoidably allocating PEs to control management rather than pure computation.

In contrast, DISCO’s workflow employs hierarchical double buffering: input matrices are tiled, staged in the SPM, then loaded to VWRs before compute, maximizing reuse. Matrix B is transposed during load to optimize access for the SIMD-capable PEs, and the specialized control units orchestrate nested loop scheduling, memory transfer, and index tracking.

Figure 3: Mapping GEMM onto DISCO—SPM and VWRs organize data for bandwidth-efficient, parallel block processing.

A direct comparison of the innermost loop on a simplified mesh reveals substantial savings in instruction count and operational complexity for DISCO. The homogeneous OE requires every PE to handle both computation and dataflow control, resulting in higher cycle overhead and many instructions spent on address calculation. In contrast, DISCO's dedicated units offload control, reducing per-PE instruction count and overall cycle cost.

Figure 4: Side-by-side mapping of matrix multiplication’s innermost loop in OE and DISCO, highlighting instruction and overhead reductions enabled by heterogeneity and SPM.

End-to-End Application: Seizure Detection Pipeline

Performance, energy, and power efficiency are rigorously assessed using a real-world, compute-bound biomedical workload: a Transformer-based seizure detector for multi-channel EEG. The pipeline (EEG→STFT→embedding→Transformer layers/attention/MHSA→classification) is profiled in detail, quantifying the computational footprint of each kernel.

Figure 5: Seizure detection transformer pipeline; computationally intensive kernels (e.g., STFT, Mmul) are efficiently mapped onto the CGRA, whereas non-linearities (GeLU, Softmax) are relegated to the CPU due to non-integer ALU limitations.

Profiling shows that after optimizing the Softmax routine (row-wise factored ConSmax approximation), over 91% of total cycles are monopolized by Mmul and STFT, precisely the kernels best matched by CGRA acceleration. Computationally intensive, non-linear kernels are executed on the CPU. The performance bottleneck is thus shifted entirely onto the dataflow and matrix operations domain.

Experimental Results and Quantitative Analysis

Synthetic Benchmarks

Evaluation on the PolyBench suite (mmul, gemm, 2mm, 3mm) highlights that the presence of SPM in DISCO reduces memory load traffic by $8\times$. This high ratio is a direct manifestation of superior data reuse. OE, without local storage, inefficiently fetches operands repeatedly, dramatically increasing off-chip accesses.

DISCO amortizes memory latency via double-buffering, masking SPM load time and maximizing PE utilization except for the smallest kernel sizes. The overhead of address computation and memory transfer in OE results in a window where up to two-thirds of cycles are spent not computing.

Figure 6: DISCO and OE performance (execution time, energy, power) across matrix multiplication benchmarks; SPM and heterogeneity substantially mitigate execution stalls and reduce energy.

When normalized for operating frequency, OE-200 MHz generally minimizes energy but at the cost of longer execution time, while OE-700 MHz obtains the best latency but suffers from high power draw. DISCO balances the two, with better performance at equivalent frequencies and reduced memory energy due to SPM locality.

Application-Level Kernels

Within the seizure detection pipeline, GEMM kernels of various sizes are mapped on the CGRAs. OE-700 achieves superior latency but at increased energy and power. DISCO pays an energy premium for smaller kernels due to the overhead of its SPM, but as size increases, its relative efficiency improves.

Figure 7: TSD pipeline matrix multiplication execution time, energy, and power metrics; trends demonstrate workload-dependent crossover between OE and DISCO.

For the STFT (FFT-based), DISCO attains the highest energy efficiency, leveraging wide intra-array pathways and butterfly-dataflow-specific hardware shuffle in the SPM, achieving up to $122 \times$ speedup over CPU for OE-700 and $113 \times$ for DISCO.

Once the full workflow is considered, both OE and DISCO vastly outperform the CPU baseline (speedups up to $29.9 \times$ at 700 MHz and $8.8 \times$ at 200 MHz, with $0.14 \times$–$0.18 \times$ the corresponding energy), and show close parity when run at equal frequency: their respective architectural strengths balance out across diverse compute and dataflow patterns.

Figure 8: Full pipeline results emphasize both the massive advantage of CGRA-based compute over CPU and show nearly balanced performance for OE and DISCO when normalized by frequency.

Comparison to State-of-the-Art CGRAs

DISCO is evaluated against a suite of academic and industrial CGRAs using a canonical FFT kernel. When factoring in both organization and frequency scaling, DISCO delivers the lowest energy per operation ($C = A \times B$0J), and achieves a $C = A \times B$1 speedup versus the most efficient contemporary architecture (R-Blocks), with significantly higher throughput per unit of power.

Implications and Future Directions

The results substantiate that heterogeneity and SPM integration deliver substantial energy and memory bandwidth improvements for data-shuffling and matrix-intensive workloads. However, homogeneity remains preferable for aggressive area constraints or applications where frequency scaling is easily afforded and data reuse is limited. Notably, the simplicity and area compactness of OE ($C = A \times B$2–$C = A \times B$3 less than competitors) enable scaling strategies that can close the performance gap given sufficient parallel instantiation.

For edge deployment, the choice of architecture is application- and envelope-dependent:

• Homogeneous, low-frequency CGRAs: optimal for miniaturized, ultra-low-power domains (e.g., implantables), where area and static power dominate.
• High-frequency, homogeneous arrays: ideal for compute- (not energy-) constrained systems or where die footprint can accommodate multi-instantiation.
• Heterogeneous, SPM-based arrays: best for throughput- and energy-constrained domains that demand high data reuse and autonomy (e.g., battery-powered wearables).

Practically, these findings offer prescriptive guidance for hardware/software co-design in energy-critical biomedical AI and signal processing on edge nodes, and hint at future directions: incorporating wider, more flexible local memory hierarchies, exploring automated mapping for complex dataflow graphs, and fine-grained heterogeneity adaptation tuned by application profiling.

Conclusion

This work rigorously quantifies the multifaceted hardware-efficiency trade-offs shaping modern CGRA deployment for edge-AI and signal processing. Local scratchpad memory and functional heterogeneity provide clear, quantifiable benefits for data-reuse and shuffling-intensive domains, while homogeneous design achieves minimal area and maximal simplicity, retaining competitiveness for certain matrices by leveraging higher clock frequencies. The selection of CGRA microarchitecture for edge applications is context-sensitive, requiring careful analysis of power, area, throughput, and workload characteristics.