MiniOpt: Reasoning to Model and Solve General Optimization Problems with Limited Resources

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Ke Zhao, Zixiang Di, Hong Qian, Xiang Shu, Yaolin Wen, Qitao Shi, Bingdong Li, Xingyu Lu, Xiangfeng Wang, Jun Zhou, Ke Tang, Yang Yu
• Venue: arXiv: cs.LG
• Published: 2026-06-24
• Collected: 2026-06-25
• URL: http://arxiv.org/abs/2606.25832v1
• PDF: https://arxiv.org/pdf/2606.25832v1
• DOI: UNKNOWN

Plain-Language Brief

Researchers propose MiniOpt, a reinforcement learning framework that helps large language models solve various optimization problems with limited resources. It breaks down optimization reasoning into structured modeling and solver generation. The team developed OptReward, a hierarchical reward function, and an optimization-oriented policy optimization strategy to enhance the efficiency of learning and stabilize reinforcement learning for compact models.

What the Document Says

• The paper discusses MiniOpt, a framework that uses reinforcement learning for language models to solve diverse optimization problems with limited resources.
• MiniOpt decomposes optimization reasoning into structured modeling and solver generation.
• OptReward is introduced as a reward function with hierarchical score structure for joint evaluation of formulation and solution.
• An optimization-oriented policy optimization strategy was developed to improve exploration efficiency and stabilize reinforcement learning for compact models.
• MiniOpt-3B showed strong optimization generalization across various optimization types, scenarios, and task domains.
• For smaller models (fewer than 10B parameters), MiniOpt series achieved the highest average solving accuracy.
• Even for larger models (more than 10B parameters), MiniOpt still demonstrated competitive performance.

Key Concepts Mentioned

• optimization problems
• large language models
• reinforcement learning
• OptReward
• policy optimization strategy

Claims Made by the Document

• MiniOpt provides an effective pathway for developing compact optimization-specialized language models.
• MiniOpt-3B exhibits strong optimization generalization across various optimization types, problem scenarios, and task domains.

Evidence Used

• The paper presents experiments showing the strong optimization generalization of MiniOpt-3B across different settings.
• The code for MiniOpt is available on GitHub.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Achieving strong optimization generalization across diverse optimization problems while requiring limited training resources remains a challenging problem for optimization-oriented large language models (LLMs). Existing approaches typically rely on large-scale supervised datasets, costly reasoning annotations, and expensive intermediate step verification, resulting in substantial training overhead. To address these challenges, we propose MiniOpt, a reinforcement learning framework that learns to solve optimization problems through an 'reasoning-to-model-and-solve' paradigm. MiniOpt decomposes optimization reasoning into structured optimization modeling and executable solver generation. Building upon this paradigm, we introduce OptReward, a reward function with hierarchical score structure that jointly evaluates formulation and solution, enabling effective policy learning without expert demonstrations. We further develop an optimization-oriented policy optimization strategy that improves exploration efficiency and stabilizes reinforcement learning for compact models. Extensive experiments show that MiniOpt-3B exhibits strong optimization generalization across various optimization types, problem scenarios, and task domains. For models with fewer than 10B parameters, MiniOpt series achieves the highest average solving accuracy (SA). For models with more than 10B parameters, MiniOpt still shows competitive performance. These results suggest that optimization-oriented reward design and reinforcement learning provide an effective pathway for developing compact optimization-specialized language models with strong optimization generalization capabilities. The code is available at https://github.com/Hsiang-1/MiniOpt.