SciWalker: Synthesizing Scientific Coding Problems with Operator Graphs and Execution Feedback

Source facts

The preprint describes a framework for synthesizing scientific coding problems by sampling operator chains from graphs built from scientific library interfaces and operation modes. An LLM generates problem statements, reference solutions, and tests; failed generations are repaired using execution feedback. The abstract reports 8,178 problems across five scientific domains and 32 subdomains, and a reinforcement-learning experiment with Qwen3.5-9B in which SciCode subproblem accuracy increased from 29.3% to 39.2%.

Analysis

This is a proposed data-engineering and evaluation path for scientific coding agents: interface-aware problem generation is coupled to executable validation rather than relying only on text-based filtering.

Uncertainty/Doubt

• This is an arXiv preprint. The dataset quality, execution coverage, and reported accuracy improvement were not independently reproduced here.