FIELD NOTES TOPIC
swe-bench
2 notes, newest first.
PAIChecker: Uncovering and Checking PR-Issue Misalignment in SWE-Bench-Like Benchmarks
Researchers present PAIChecker, a system designed to detect misalignment between Pull Requests (PRs) and Issues in SWE-bench-like benchmarks used for evaluating Large Language Models (LLMs). The system identifies…
PAIChecker: Uncovering and Checking PR-Issue Misalignment in SWE-Bench-Like Benchmarks
Researchers have developed a system called PAIChecker to check for misalignment between Pull Requests (PRs) and issues in software development benchmarks used to evaluate the issue resolution capability of large…
