Harness Engineering for LLM-Driven GPU Kernel Generation

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Yue Shui, Chenyu Ma, Hangfei Xu, Shengzhao Wen, Yanpeng Wang
• Venue: arXiv: cs.LG
• Published: 2026-07-20
• Collected: 2026-07-21
• URL: http://arxiv.org/abs/2607.17979v1
• PDF: https://arxiv.org/pdf/2607.17979v1
• DOI: UNKNOWN

Plain-Language Brief

The paper discusses a system for optimizing GPU kernels using large language models (LLMs). The system separates an evaluation harness from a profile-backed optimization controller to enforce compilation, correctness, timing, and artifact archival. Human-authored skills are used to capture operator constraints, references, profiling procedures, and promotion rules, while Codex and Claude Code agents generate candidate kernels within these constraints. The system was tested in the MLSys 2026 FlashInfer AI Kernel Generation Contest and outperformed the Full-Agent artifacts across evaluated definitions.

What the Document Says

• The paper proposes a harness-centered system for LLM-driven GPU kernel optimization.
• The system separates an evaluation harness from a profile-backed optimization controller.
• The harness enforces compilation, correctness, official-aligned timing, and artifact archival.
• The controller turns profiler and workload evidence into bounded candidate-generation decisions.

Key Concepts Mentioned

• Large language models (LLMs)
• GPU kernel generation
• MLSys 2026 FlashInfer AI Kernel Generation Contest
• NVIDIA Blackwell B200 GPUs
• evaluation harness
• profile-backed optimization controller
• Codex
• Claude Code

Claims Made by the Document

• Generated code can be reliably constrained, validated, profiled, and selected for practical effectiveness.
• The system improves mean-latency speedups over supplied FlashInfer baselines.

Evidence Used

• Across five operator definitions, the retained official-aligned artifacts achieved mean-latency speedups over supplied FlashInfer baselines of 1.62x, 18.05x, 29.68x, 1.12x, and 13.70x.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Large language models (LLMs) can assist GPU kernel generation, but their practical effectiveness depends on whether generated code can be reliably constrained, validated, profiled, and selected. This paper presents a harness-centered system for LLM-driven GPU kernel optimization in the MLSys 2026 FlashInfer AI Kernel Generation Contest on NVIDIA Blackwell B200 GPUs. The system separates an evaluation harness from a profile-backed optimization controller: the harness enforces compilation, correctness, official-aligned timing, and artifact archival, while the controller turns profiler and workload evidence into bounded candidate-generation decisions. Human-authored skills capture operator constraints, references, profiling procedures, and promotion rules, while Codex and Claude Code agents generate candidate kernels inside those constraints. Across five operator definitions, the retained official-aligned artifacts achieved mean-latency speedups over supplied FlashInfer baselines of 1.62x, 18.05x, 29.68x, 1.12x, and 13.70x. The Agent-Assisted kernels outperform the Full-Agent artifacts across the evaluated definitions, indicating that expert-provided optimization directions, high-quality references, and workload context remain critical for reliable AI-driven kernel optimization.