Field notes
2,343 notes from my AI research vault, mapped by topic. Pick a topic to see its notes, or search.
Click the web, then scroll to zoom
Every note is in the list below. The web of topics draws here once the page has loaded.
26 of 2,343 notes match.
- Question answered
- Source note
- Topic
- Shared notes
Notes on software engineering
26 notes, page 1 of 2
ParanoiaEval: Benchmarking Unnecessary Defensive Work in Agentic Coding
The preprint introduces a benchmark for unified evaluation of risk-treatment capabilities in coding agents. It maps Avoidance, Transfer, Mitigation, and Acceptance from software risk management to 200…
Shockingly Simple Self-retrospection Improves Agentic Models Without RL
The preprint introduces Retrospection-Only Fine-Tuning (ROFT), an online procedure in which an agent attempts a task, observes feedback, generates a retrospective explanation, and fine-tunes only on the explanation…
CodeMidas: Scaling Agentic Coding RL Environments from Code Itself
CodeMidas converts implemented functionality in open-source codebases into executable reinforcement-learning environments using source code as the only task-specific input. Agents formulate behavioral specifications…
An Empirical Study of Harness Design for Coding Agents
The paper varies planning, action space, and context management in a fixed coding harness and evaluates 176 matched settings across four models on SWE-Bench Verified and Terminal-Bench 2.1. It reports that context…
Repo0: Design-Driven Zero-to-All Code Generation
The paper introduces Repo0, a framework for zero-to-all code generation that maintains an explicit architectural state using a Dual-Directed Acyclic Graph (Dual-DAG). It iteratively evolves component boundaries guided…
VICBench: A Multi-Language Benchmark for Code Vulnerability Detection
The paper introduces VICBench, a new benchmark dataset for evaluating code vulnerability detection tools. VICBench contains 100 verified vulnerability-inducing commits (VICs) across 88 projects in Python, Java, and…
VICBench: A Multi-Language Benchmark for Code Vulnerability Detection
Researchers have created a benchmark dataset called VICBench for evaluating code vulnerability detection tools. The dataset contains 100 verified vulnerabilities across 88 projects in Python, Java, and C++, covering 48…
RepoProbe: Benchmarking Architecture-Aware Repository Comprehension with Checklists
The text discusses a new benchmark called RepoProbe for evaluating the ability of Large Language Models (LLMs) to understand repository-scale code. The benchmark uses open-ended questions from GitHub Discussions and…
Agentic Method for Deterministic Validation of Legacy Code Migration
The paper proposes a new method called 'Locksmith Loop' for testing the migration of COBOL programs to Java. This method is designed to improve coverage beyond input-search plateaus, reaching near complete coverage in…
From Resource Flow to Executable Tests: Petri-Net-Guided LLM Test Generation for Concurrent Stateful Rust APIs
The authors present a methodology using Petri nets to generate tests for concurrent stateful Rust APIs, aiming to bridge the gap between formal scenario design and low-cost test concretization.
From Resource Flow to Executable Tests: Petri-Net-Guided LLM Test Generation for Concurrent Stateful Rust APIs
The authors propose a methodology for generating executable tests for concurrent stateful Rust APIs using Petri nets. This approach aims to bridge the gap between formal scenario design and low-cost test…
From Resource Flow to Executable Tests: Petri-Net-Guided LLM Test Generation for Concurrent Stateful Rust APIs
The paper presents a methodology using Petri nets for test generation in concurrent stateful Rust APIs, aiming to bridge the gap between formal scenario design and low-cost test concretization. It allows large language…
Multi-stage Dynamic Selection for Cross-Project Defect Prediction
The paper proposes a new framework for Cross-Project Defect Prediction (CPDP) that addresses distribution shift issues between training and target projects by using a two-stage multiple classifier system selection…
TRIM: Reducing AI-Generated CodeSlop via Agent Trajectory Minimization
The research presented in 'TRIM: Reducing AI-Generated CodeSlop via Agent Trajectory Minimization' aims to address the issue of verbose and redundant code generated by coding agents, which are increasingly used for…
Early Adoption of Agentic Coding Tools by GitHub Projects
The paper investigates the early adoption of automated coding tools by GitHub projects. It found that while intensive adoption is concentrated in a few projects, smaller projects tend to have higher participation…
Early Adoption of Agentic Coding Tools by GitHub Projects
This research investigates the adoption and management of agentic coding tools in software development on GitHub. The study analyzes 25,264 agentic pull requests from 2,361 popular repositories to understand (1) how…
ProjAgent: Procedural Similarity Retrieval for Repository-Level Code Generation
The paper proposes ProjAgent, a system for generating code at the repository level. It introduces 'procedural similarity' as a new way to retrieve relevant functions, addressing a limitation of existing methods that…
ProjAgent: Procedural Similarity Retrieval for Repository-Level Code Generation
The paper presents ProjAgent, a system for generating code at the repository level that uses 'procedural similarity' as a new signal in the retrieval process. This system decomposes the target function into…
Harnessing Code Agents for Automatic Software Verification
The authors propose a new method for automatic software verification using large language models (LLMs) in the form of 'code agents'. Instead of wireting a fixed proof strategy, they allow the code agent to choose its…
RuBench: A Repository-Level Agentic Coding Benchmark with Natively Authored Russian Task Specifications
The author introduces RuBench 1.0, a benchmark for evaluating coding agents that use native language tasks instead of English. The benchmark includes 25 tasks mined from five open-source repositories and is judged by…
TestEvo-Bench: An Executable and Live Benchmark for Test and Code Co-Evolution
The paper introduces TestEvo-Bench, a new benchmark for testing software that evolves with its code. This benchmark is designed to evaluate how well a test automation agent understands the impact of a code change on…
Are Performance-Optimization Benchmarks Reliably Measuring Coding Agents?
The paper evaluates three repository-level performance optimization benchmarks (GSO, SWE-Perf, and SWE-efficiency) used to measure coding agents. The authors find issues with the stability of runtime for these…
LLVM-Bench: Benchmarking and Advancing Large Language Models for LLVM Compiler Issue Resolution
The authors present LLVM-Bench, a large-scale benchmark for testing the effectiveness of large language models (LLMs) in resolving issues related to the LLVM compiler. They found that current LLM-based techniques are…
SWE-Doctor: Guiding Software Engineering Agents with Runtime Diagnosis from Multi-Faceted Bug Reproduction Tests
Researchers propose a new software agent called SWE-Doctor that uses multiple bug reproduction tests (BRTs) to generate more effective patches for resolving software issues. The agent first generates multi-faceted…
An AI research agent drafts these notes from public papers and posts, and each one keeps its source link. Only the folders I choose to publish leave my Obsidian vault.
