Harness Engineering for LLM-Driven GPU Kernel Generation

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Yue Shui, Chenyu Ma, Hangfei Xu, Shengzhao Wen, Yanpeng Wang
• Venue: arXiv: cs.LG
• Published: 2026-07-20
• Collected: 2026-07-21
• URL: http://arxiv.org/abs/2607.17979v1
• PDF: https://arxiv.org/pdf/2607.17979v1
• DOI: UNKNOWN

Plain-Language Brief

This research presents a system that uses large language models (LLMs) to optimize GPU kernels. The system separates an evaluation harness from an optimization controller, with human-authored skills and code agents generating candidate kernels within constraints. In the MLSys 2026 FlashInfer AI Kernel Generation Contest, the system achieved mean-latency speedups over baseline kernels for five operator definitions.

What the Document Says

• The research discusses a harness-centered system for GPU kernel optimization using large language models (LLMs).
• The system consists of an evaluation harness and a profile-backed optimization controller.
• The harness enforces compilation, correctness, timing, and archival of artifacts.
• The controller uses profiler and workload evidence to make bounded candidate-generation decisions.
• Human-authored skills provide operator constraints, references, profiling procedures, and promotion rules.
• Codex and Claude Code agents generate candidate kernels within these constraints.
• In the MLSys 2026 FlashInfer AI Kernel Generation Contest on NVIDIA Blackwell B200 GPUs, the system achieved speedups over baseline kernels for five operator definitions.
• The retained official-aligned artifacts outperformed supplied FlashInfer baselines by 1.62x, 18.05x, 29.68x, 1.12x, and 13.70x in terms of mean-latency speedups.

Key Concepts Mentioned

• Large Language Models (LLMs)
• GPU Kernel Optimization
• Harness
• Controller
• Codex
• Claude Code
• MLSys 2026 FlashInfer AI Kernel Generation Contest
• NVIDIA Blackwell B200 GPUs

Claims Made by the Document

• The system can assist in GPU kernel generation.
• The effectiveness of the system depends on reliable constraint, validation, profiling, and selection of generated code.

Evidence Used

• The research discusses a system for LLM-driven GPU kernel optimization.
• The system's components and functions are detailed throughout the text.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Large language models (LLMs) can assist GPU kernel generation, but their practical effectiveness depends on whether generated code can be reliably constrained, validated, profiled, and selected. This paper presents a harness-centered system for LLM-driven GPU kernel optimization in the MLSys 2026 FlashInfer AI Kernel Generation Contest on NVIDIA Blackwell B200 GPUs. The system separates an evaluation harness from a profile-backed optimization controller: the harness enforces compilation, correctness, official-aligned timing, and artifact archival, while the controller turns profiler and workload evidence into bounded candidate-generation decisions. Human-authored skills capture operator constraints, references, profiling procedures, and promotion rules, while Codex and Claude Code agents generate candidate kernels inside those constraints. Across five operator definitions, the retained official-aligned artifacts achieved mean-latency speedups over supplied FlashInfer baselines of 1.62x, 18.05x, 29.68x, 1.12x, and 13.70x. The Agent-Assisted kernels outperform the Full-Agent artifacts across the evaluated definitions, indicating that expert-provided optimization directions, high-quality references, and workload context remain critical for reliable AI-driven kernel optimization.