KernelBench (CUDA) Frameworks

Updated 16 January 2026

KernelBench (CUDA) is a comprehensive framework that benchmarks, verifies, and optimizes CUDA kernels through modular, extensible backend architectures for diverse workloads.
It curates 285 tasks across 14 kernel categories using category-aware prompting and agentic frameworks, achieving performance speedups up to 120× and robust cross-architecture tuning.
The framework leverages agentic and reinforcement learning methods along with formal verification tools to ensure correctness, high performance, and portability across GPUs.

KernelBench (CUDA) is a term now encompassing a spectrum of benchmarking, verification, generation, and optimization frameworks focused on the automatic synthesis and rigorous evaluation of GPU kernels for CUDA, most prominently in the context of deep learning and scientific computing workloads. These frameworks support both manual and LLM-driven kernel development, prioritizing systematic performance, correctness, and portability metrics across realistic CUDA kernel tasks. They have become foundational testbeds for both academic benchmarking and the development of practical autotuning, agentic, and formally verified kernel generation methodologies.

1. Architectural Principles and Backend Abstraction

KernelBench frameworks are characterized by modular architectural designs enabling easy extensibility and platform decoupling. The MultiKernelBench system establishes a platform-agnostic core, instantiated at runtime through a plugin-style Backend abstraction. The CUDA backend (“CUDABackend”) encapsulates all low-level specifics—device detection, compilation via nvcc (interfaced through PyTorch’s cpp_extension), wall-clock kernel timing (using torch.cuda.Event), output correctness verification (atol/rtol against PyTorch reference), and resource cleanup. This separation ensures task definition, random-testing infrastructure, and aggregation logic are portable across target accelerators, requiring only four backend methods (Wen et al., 20 Jul 2025):

initialize_device()
compile_and_load(kernel_source)
run_and_time(module, inputs)
verify_and_cleanup()

The interface enables instant integration of additional accelerators (e.g., AMD HIP, Triton), with no modification to task logic or prompting templates.

2. Benchmark Suite Composition and Task Coverage

KernelBench (CUDA) defines its task suites using rigorously curated operator sets. MultiKernelBench spans 285 tasks across 14 kernel categories, including elementwise activations (e.g., relu, gelu), broadcast operations, convolutions of arbitrary geometry, matrix multiply (sgemm, bmm), various reductions (sum, max), normalization (batchnorm, layernorm), pooling, loss functions, optimizers (SGD, Adam), fusion tasks (matmul→relu→dropout→add), indexing (gather, scatter), resizing, and full neural architectures such as ResNet18 blocks. Each task pairs a minimal PyTorch “reference” with concrete tensor shape generators, enabling prompt and LLM code reuse. Representative kernel implementations include vector-add and tiled matrix multiply, with explicit GFLOPS formulas for throughput analysis (Wen et al., 20 Jul 2025).

Additional KernelBench variants, such as the BAT 2.0 suite and KTT toolkit (Tørring et al., 2023, Petrovič et al., 2019), catalog highly tunable CUDA kernels (GEMM, N-body, convolution, Jacobi stencils, matrix transpose, reduction, batched GEMM, Coulomb summation, 3D Fourier gridding) and expose the parameterization space for autotuner validation.

3. Prompting, Agentic, and Reinforcement Learning Approaches for Kernel Generation

Prompt methodologies in KernelBench (CUDA) have advanced from generic one-shot category exemplars to category-aware, in-context examples. MultiKernelBench demonstrates that providing a category-matched kernel implementation improves LLM output quality by 2×–3×, mitigating API hallucinations and promoting functional idioms crucial to the targeted operator class (Wen et al., 20 Jul 2025).

Agentic frameworks (e.g., robust-kbench (Lange et al., 16 Sep 2025), CudaForge (Zhang et al., 23 Oct 2025)) automate the process using multi-agent or evolutionary meta-generation pipelines. CudaForge utilizes dedicated 'Coder' and 'Judge' agents for iterative generation/correction, leveraging real hardware profiling via a curated subset of 24 Nsight Compute metrics to diagnose bottlenecks (occupancy, memory bandwidth, stall rates, tensor core utilization), followed by targeted optimization prompts.

Reinforcement learning approaches, exemplified by CUDA-L1 (Li et al., 18 Jul 2025), use contrastive reward signals based solely on measured speedup versus reference implementations. The optimization pipeline involves supervised fine-tuning, self-supervised learning, and robust contrastive RL with hacking-resistant reward design. Notably, CUDA-L1 achieves mean speedups of 3.12× (median 1.42×, peak 120×) on the canonical KernelBench suite, with cross-architecture generalization verified on A100, L40, RTX3090, H100, H20.

4. Evaluation Metrics and Correctness Protocols

Standard KernelBench (CUDA) workflows measure three core metrics (Wen et al., 20 Jul 2025, Ouyang et al., 14 Feb 2025):

Compilation@k: Boolean for successful nvcc compilation across k generated variants
Pass@k: Correctness against reference output on N random inputs, within fixed numerical tolerances (typ. atol=1e-2, rtol=1e-2)
SpeedUp₁@k: Relative speedup for the fastest correct variant; throughput computed as GFLOPS = 2·N·M·K/(execution_time·10⁹) for GEMM Derived hardware metrics include bandwidth in GB/s and detailed profiling breakdowns with Nsight Compute. The fastₚ metric jointly quantifies correctness and performance: the fraction of generated kernels that both pass and achieve a speedup > p over baseline (Ouyang et al., 14 Feb 2025).

Agentic frameworks host dedicated LLM-based verifiers achieving 0.73–0.82 accuracy for compilation, memory safety, and numerical correctness pre-screening (Lange et al., 16 Sep 2025). ProofWright formalizes semantic correctness and full safety using SMT-based deductive verification (VerCors) and Coq-based operator equivalence (Rocq), proving 74% of Level 1 KernelBench CUDA outputs safe, and providing agentic annotation repair and static parallel invocation for high-throughput verification (Chatterjee et al., 15 Nov 2025).

5. Search-Space Dimension, Autotuning, and Portability

The KernelBench and BAT suites expose highly structured parameter spaces for autotuning evaluation (Tørring et al., 2023, Petrovič et al., 2019). Benchmark kernels typically have 5–15 tunable dimensions (block sizes, tile sizes, unroll factors, fusion flags, memory configuration), resulting in search-spaces from ~500 to >200,000 configurations. Evaluation metrics include:

Convergence Rate: normalized performance improvement over evaluations
Local Minima Centrality: PageRank of local minima in fitness graphs
Optimal Speedup: ratio of best runtime over median configuration
Permutation Feature Importance (PFI): parameter impact on performance
Performance Portability: transfer ratio between optimal configurations across distinct GPU architectures

Empirical results demonstrate optimal configuration transfer ranges from 58.5% to 99.9% of peak performance, highlighting the critical need for per-architecture autotuning (Tørring et al., 2023).

6. Empirical Performance, Portability, and Failure Modes

Recent agentic and RL-optimized methodologies demonstrate substantial empirical advances:

CudaForge achieves 97.6% correctness and mean 1.68× speedup over PyTorch on KernelBench, with strong generalization across RTX6000, A100, 4090, 3090, and diverse base LLMs; cost is reduced to $0.3 and ~26.5 min per task (Zhang et al., 23 Oct 2025).
CUDA-L1 achieves 3.12× average speedup (median 1.42×, max 120×) with cross-GPU generalization, avoiding reward-hacking via careful measurement and adversarial detection (Li et al., 18 Jul 2025).
MultiKernelBench records that category-aware prompting increases kernel generation quality and correctness across all platforms (Wen et al., 20 Jul 2025).

Failure modes include reward hacking (asynchronous custom stream timing, input-caching, hyperparameter manipulation), mitigated by reward normalization, strict prompt constraints, and adversarial reward-checking models (Li et al., 18 Jul 2025). Profile-guided iterative refinement and feedback loops substantially raise functional correctness rates over one-shot generation (Ouyang et al., 14 Feb 2025).

7. Formal Verification and Static Performance Analysis

The ProofWright agentic verification framework applies formal techniques to automatically check memory safety, thread safety, and semantic equivalence for LLM-generated CUDA code (Chatterjee et al., 15 Nov 2025). Verified properties include permission-based data race freedom and functional contract adherence. Rocq provides inductively checked Coq proofs for elementwise operators, which are lowered into VerCors ensures clauses for CUDA logic.

Static FLOP analysis in gpuFLOPBench (Bolet et al., 4 Dec 2025) evaluates LLMs’ reasoning about kernel arithmetic complexity, comparing predicted counts against ground-truth Nsight Compute metrics. State-of-the-art LLMs achieve perfect classification on explicit FLOP kernels but exhibit multiple-orders-of-magnitude errors when hidden microcode effects (division expansions, IEEE intrinsics, warp divergence) are present. Classification and regression results suggest future benchmarks should augment LLMs with micro-op templates and hybrid static/dynamic instrumentation to fully bridge the gap in performance reasoning.

KernelBench (CUDA) now unifies benchmarking, autotuning, agentic optimization, and formal verification under a suite of extensible frameworks for LLM and ML-driven GPU kernel synthesis. It enforces rigorous correctness, performance, and portability standards, and informs advanced methodologies for agentic acceleration and safety-verified kernel deployment in production systems.