Source note2 min read
Co-Evolving LLM Evaluators and Policies via DynamicRubric
Source
arXiv
arxiv.org/abs/2607.20083v1
Source
- Source name: arXiv
- Source kind: paper
- Authors / organization: Beining Wang, Weihang Su, Hongtao Tian, Hao Kong, Tao Yang, Ting Yao, Qingyi Pan, Yueyue Wu, Qingyao Ai, Min Zhang, Yiqun Liu
- Venue: arXiv: cs.LG
- Published: 2026-07-22
- Collected: 2026-07-23
- URL: http://arxiv.org/abs/2607.20083v1
- PDF: https://arxiv.org/pdf/2607.20083v1
- DOI: UNKNOWN
Plain-Language Brief
The paper introduces DynamicRubric, a framework that co-evolves language model evaluators and policies by generating weighted binary rubrics for each candidate set and aggregating the resulting judgments into response-level scores. The method improves evaluator performance and provides stronger policy supervision compared to baselines in various tasks, including verifiable reasoning and coding tasks. It is currently being used in WeChat Search's AI answering scenario.
What the Document Says
- The paper presents DynamicRubric, a framework for co-evolving evaluators and policies of large language models.
- It generates weighted binary rubrics for each candidate set and aggregates the resulting judgments into response-level scores.
- Experiments with 8B backbones show that DynamicRubric improves evaluator performance and provides stronger policy supervision than baselines using a 70B reward model or a 235B static rubric generator.
- DynamicRubric-optimized policies exhibit gains on verifiable reasoning and coding tasks.
- A DynamicRubric-optimized model is deployed in WeChat Search's AI answering scenario, serving all online traffic across tens of millions of requests per day and improving key online metrics.
- The paper suggests that evaluators should evolve with the policies they supervise.
Key Concepts Mentioned
- DynamicRubric
- language model evaluators
- policies
- binary rubrics
- response-level scores
- WeChat Search
Claims Made by the Document
- DynamicRubric improves evaluator performance.
- DynamicRubric provides stronger policy supervision compared to baselines.
- DynamicRubric-optimized policies show gains on verifiable reasoning and coding tasks.
- A DynamicRubric-optimized model is deployed in WeChat Search's AI answering scenario.
Evidence Used
- Improvements observed in experiments with 8B backbones
- Results from using a 70B reward model or a 235B static rubric generator as baselines
- Gains on verifiable reasoning and coding tasks
- Deployment in WeChat Search's AI answering scenario.
Methods / Approach
UNKNOWN
Limitations Stated by the Authors
UNKNOWN
Limitations Noted by the Agent
Agent interpretation — not from the source.
UNKNOWN
Open Questions
UNKNOWN
Notes for Future Q&A
May include interpretation. Labeled where it does.
UNKNOWN
Raw Abstract / Excerpt
Post-training with evaluator feedback on policy-induced samples serves as a major mechanism for improving large language models. As policies improve, these sampled responses become close in quality. These close candidates create a bottleneck for policy optimization: collapsed relative evaluator score gaps yield weak or misleading policy supervision. We theoretically characterize why these gaps matter through a probability allocation view, showing that the directional gain of shifting probability mass from one response to another is exactly the evaluator score gap between them. This identifies relative score gaps as the policy optimization signals that guide updates. Motivated by this view, we propose DynamicRubric, a response-set-conditioned evaluator--policy co-evolution framework that generates weighted binary rubric items for each candidate set and aggregates the resulting judgments into response-level scores. In our experiments with 8B backbones, DynamicRubric improves evaluator performance and provides stronger policy supervision than baselines using a 70B reward model or a 235B static rubric generator. DynamicRubric-optimized policies also show gains on verifiable reasoning and coding tasks. A DynamicRubric-optimized model is fully deployed in WeChat Search's AI answering scenario, where it serves all online traffic across tens of millions of requests per day and improves key online metrics. These results suggest a principle for evaluator-guided post-training: evaluators should evolve with the policies they supervise.
