A$^{2}$utoLPBench: An Auto-Generated, Agent-Friendly LP Benchmark via Inverse-KKT Construction

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Shuo Ren, Yaohui Han, Yifan Shi, Libo Shen, Haodong Lu, Dongfang Wu, Rongliang Fu, Bei Yu, Tsung-Yi Ho
• Venue: arXiv: cs.AI
• Published: 2026-07-02
• Collected: 2026-07-03
• URL: http://arxiv.org/abs/2607.02141v1
• PDF: https://arxiv.org/pdf/2607.02141v1
• DOI: UNKNOWN

Plain-Language Brief

The paper introduces A$^{2}$utoLPBench, a new linear programming (LP) benchmark for testing LLM-driven agents. The benchmark generates LP problems in plain text, and uses a known feasible point and dual to write down a problem where the answer is already known by construction. This eliminates the need for solver calls or human annotators. The evaluation environment includes a reference solver-critic baseline and instructions written for LLM-driven agents. A key advantage of this benchmark is that it can generate an unlimited supply of fresh problems, has adjustable difficulty, and resists training data leakage.

What the Document Says

• A$^{2}$utoLPBench is a new LP benchmark for testing LLM-driven agents.
• The benchmark generates LP problems in plain text using a known feasible point and dual.
• Answers are known by construction with no solver call or human annotator.
• The evaluation environment includes a reference solver-critic baseline and instructions for LLM-driven agents.
• A key advantage is that the benchmark can generate an unlimited supply of fresh problems, has adjustable difficulty, and resists training data leakage.

Key Concepts Mentioned

• linear programming (LP)
• LLM-driven agents
• feasible point
• dual
• benchmark

Claims Made by the Document

• A$^{2}$utoLPBench is a new LP benchmark for testing LLM-driven agents.
• The benchmark can generate an unlimited supply of fresh problems.
• The difficulty of the problems can be adjusted by $(n,m)$.
• Answers are known by construction with no solver call or human annotator.
• The evaluation environment includes a reference solver-critic baseline and instructions for LLM-driven agents.

Evidence Used

• The text describes the creation of A$^{2}$utoLPBench as a generator rather than a fixed dataset.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Most LP-from-text benchmarks are static datasets of word problems written and labeled by hand. Once such a dataset is released, its size is fixed, its difficulty is fixed, and every problem can leak into the training data of future LLMs. We present \textbf{A$^{2}$utoLPBench}, a benchmark for testing LLM-driven agents on linear programming problems written in plain text. We first pick a feasible point and dual, then write down a problem for which that point is optimal and the objective value is known. The answer is known by construction, with no solver call and no human annotator. The evaluation environment bundles a reference solver-critic baseline and a Docker image whose usage instructions are written for an LLM-driven agent to read. With these in place, any agent can run the benchmark and get a calibrated score with one command. Because the benchmark is a generator rather than a fixed dataset, it has properties no fixed dataset can match: an unlimited supply of fresh problems, a difficulty knob set by $(n,m)$, ground-truth answers correct by construction, low LLM-side cost per problem relative to human authoring, repeatable scores across independent batches, and resistance to training-data leakage when fresh post-cutoff seed ranges are used.