Exploiting pre-optimized kernels with polyhedral transformations for CGRA compilation

Abstract: Modern computing workloads commonly involve matrix-matrix multiplication (mmul) as a core computing pattern. Coarse-Grained Reconfigurable Arrays (CGRAs) can flexibly and efficiently support it, since they combine operation-level reconfigurability and high energy efficiency. However, mapping computational kernels that include mmul with state-of-the-art compilation strategies often leads to suboptimal results, since its multi-dimensional structure hampers the uncovering of its inherent parallelism and, ultimately, runtime performance. Here, we take a different position: we introduce a specialized mmul CGRA kernel schedule, parametrizable across different CGRA sizes. Then, we describe a novel compilation methodology that adapts program representations to effectively leverage it, employing polyhedral transformations to analyze complex computational patterns and expose hidden mmul operations through loop reordering and splitting. The identified patterns are then substituted with optimized assembly, while the remaining program sections are compiled independently. CGRA configurations are then generated, encompassing pre-compiled and compiled parts. Our strategy maximizes resource utilization and ultimately run-time performance, even when mmul is not directly apparent in the source code. The experimental results show speedups up to 9.1x across different benchmarks that contain hidden mmuls and CGRA instances of various sizes.

• The paper demonstrates a hybrid approach that decouples mmul regions via polyhedral transformations to substitute pre-optimized kernels, improving CGRA performance.
• The method leverages MLIR, affine dialect conversion, and SMT solving to isolate mmul subspaces for efficient kernel integration.
• Empirical results show speedups up to 9.1× on CGRA architectures, highlighting enhanced scalability and reduced compilation time compared to standard modulo scheduling.

Exploiting Pre-Optimized Kernels with Polyhedral Transformations in CGRA Compilation

Introduction and Motivation

This paper presents a compilation strategy targeting Coarse-Grained Reconfigurable Arrays (CGRAs) by hybridising domain-specific, pre-optimized matrix multiplication (mmul) kernels with general-purpose polyhedral compilation. CGRAs are compelling for edge deployments due to their high energy efficiency and reconfigurability. However, current compilation flows inadequately leverage the inherent parallelism of computational patterns such as mmul, especially when mmul is embedded within complex program hierarchies. Modulo Scheduling (MS), while standard, is fundamentally unable to exploit parallelism across loop nests, yielding suboptimal runtimes and resource utilization.

The approach advocates for decoupling the compilation of mmul regions from generic code regions. By employing polyhedral transformations to extract mmul subspaces, the compiler is able to substitute these with highly efficient pre-compiled kernels. Residual non-mmul code is compiled via standard control-dataflow graph (CDFG)-based MS compilation, preserving broad applicability.

Methodology: Polyhedral Transformation-Driven Kernel Extraction

The framework, dubbed Kernel-CGRA, consists of a three-stage pipeline. The front-end parses C into an affine dialect via MLIR, rendering loop and memory structure explicit. The middle-end applies linearity-based fusion and dependence-preserving statement reordering/splitting, formalized in the polyhedral model. Constraints are encoded into an SMT problem (solved via Z3) to identify and isolate mmul kernels in the iteration space while guaranteeing legality with respect to RAW, WAR, and WAW dependencies.

Matched regions are rewritten as calls to a parametric mmul kernel template. The kernel implementation uses output-stationary tiling mapped to CGRA tiles, with aggressive operand/data sharing and pre-computed address streams to minimize dynamic instruction counts. Registers store loop invariants, and data is broadcast as required, maximizing in-place reuse. Assembly is generated per kernel instance, parameterized by tile size, operand base addresses, and loop bounds. Live variable analysis ensures correct handoff between pre-compiled and compiled segments, with live values spilled across kernel boundaries if necessary.

Non-mmuls are reduced to SSA-based CDFGs for compilation using conventional time/space modulo scheduling. The integration is seamless: kernel and non-kernel code are merged into a unified CGRA configuration.

Strong Claims and Empirical Results

A robust experimental study is conducted on the OpenEdgeCGRA—a generic 2D torus-connected PE array—and compared to a state-of-the-art MLIR-based CGRA compiler (Compigra) employing both MS and unrolling.

Key empirical findings:

• The hybrid approach delivers consistent runtime speedups up to 9.1× relative to pure MS-based compilation.
• Compilation time is often reduced, especially for linear-algebra workloads, due to reuse of pre-compiled kernels and reduced search space in mapping.
• The scalability advantage increases with CGRA tile size and matrix dimensions; speedup increases as the problem scales.
• Benchmarks with host-accelerator switching (CPU+systolic array) or generic e-GPU exhibit inferior performance: the kernel-CGRA approach yields 9.2×–15.1× speedup over edge GPUs and 4.8×–7.1× over systolic-CPU co-execution.
• Minimal overhead is observed for applications with significant non-mmuls, with all-safe liveness management at kernel boundaries.

Theoretical and Practical Implications

The approach conclusively demonstrates that polyhedral transformations are not only essential for canonical scheduling and locality optimization but can be elevated to robustly extract domain-specific, parameterized subspaces for replacement by hand-optimized computation kernels. This represents a strong refutation of the "all-automated" philosophy of MS on fine- or medium-grained accelerators, showing that hybridization delivers tangible performance and productivity gains.

By exposing latent mmul structure—even when obfuscated by control flow or pre/post-processing stages—the compiler brings the best practices of BLAS-style library substitution (as exemplified by MKL or cuBLAS in the CPU/GPU world) to CGRAs. However, the separation is now algorithmically driven, not reliant on user annotation or pattern matching.

Scalability is inherently improved: as CGRA tile counts and matrix dimensions trend upward, the kernel approach continues to harvest parallelism and locality, with pre-compilation amortizing front-end costs. The flexible liveness and glue logic supports arbitrary mixtures of kernel and non-kernel code, enabling practical deployment for real-world pipelines beyond toy kernels.

Future Perspectives

Future work is anticipated in two key directions:

• Extension to other domain kernels: The framework is generic in its use of polyhedral analysis; other structured compute kernels (convolution, FFT, tensor contractions) may be similarly extractable and substitutable.
• Dynamic kernel/adaptive resource scheduling: As CGRA architectures grow in heterogeneity, a runtime or JIT kernel selection mechanism parameterized by data shape, hardware occupancy, and memory hierarchy may be leveraged.

Improved fusion/tiling algorithms and cost functions for polyhedral splitting can further increase coverage and efficiency, particularly for semi-regular or sparsified linear algebra.

Conclusion

By fusing pre-optimized, parameterized mmul kernels with polyhedral extraction and standard compilation, this work achieves an effective bridge between hand-tuned and fully-automated CGRA compilation. The empirical results, both in compilation and runtime, strongly support the advocated methodology. The approach constitutes a validated path towards scalable, high-efficiency deployment of mixed-structure edge workloads on CGRA platforms, with potential applicability to a wider class of structured compute kernels in future accelerator-rich architectures.