VICBench: A Multi-Language Benchmark for Code Vulnerability Detection

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Jin Lu, Xuening Han, Yang Zhong, Lin Tan, Kevin Luo, Andrew Gacek, Neha Rungta
• Venue: arXiv: cs.CR
• Published: 2026-08-12
• Collected: 2026-08-13
• URL: http://arxiv.org/abs/2608.12246v1
• PDF: https://arxiv.org/pdf/2608.12246v1
• DOI: UNKNOWN
• Registry group: academic_apis_and_preprints
• Evidence role: canonical_academic_index
• Language: en
• Country / region: UNKNOWN

Plain-Language Brief

The paper introduces VICBench, a new benchmark dataset for evaluating code vulnerability detection tools. VICBench contains 100 verified vulnerability-inducing commits (VICs) across 88 projects in Python, Java, and C++, covering 48 Common Weakness Enumeration (CWE) types. The dataset features complex real-world vulnerability fixes and VICs that are significantly larger than previous datasets.

What the Document Says

• The paper presents a new benchmark dataset called VICBench for evaluating code vulnerability detection tools.
• VICBench consists of 100 verified VICs across 88 projects in Python, Java, and C++.
• The dataset covers 48 Common Weakness Enumeration (CWE) types.
• VICBench contains complex real-world vulnerability fixes averaging 38.6 lines and corresponding VICs of 252.5 lines.
• Previous work's VIC datasets have limited language coverage, restricted patch complexity, and narrow project scope.

Key Concepts Mentioned

• VICBench
• code vulnerability detection
• benchmark dataset
• vulnerability-inducing commits (VICs)
• Common Weakness Enumeration (CWE) types

Claims Made by the Document

• VICBench enables robust evaluation of vulnerability detection approaches.
• State-of-the-art algorithms V-SZZ and LLM4SZZ achieve only 33.3%-40.1% F1 on VICBench.

Evidence Used

• Through our dual annotation by human experts and an agentic workflow, we create a benchmark - VICBench ... Our evaluation shows that state-of-the-art algorithms V-SZZ and LLM4SZZ achieve only 33.3%-40.1% F1.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Evaluating security vulnerability detection tools requires benchmark datasets with vulnerability-inducing commits (VICs) - the commits that first introduce vulnerabilities into codebases. VICs are essential for determining the full range of vulnerable software versions. Existing vulnerability datasets suffer from limited programming language coverage, restricted patch complexity, and narrow project scope. Through our dual annotation by human experts and an agentic workflow, we create a benchmark - VICBench - of 100 verified VICs for 100 CVEs across 88 projects in Python, Java, and C++, covering 48 CWE types. VICBench features complex real-world vulnerability fixes averaging 38.6 lines and corresponding VICs of 252.5 lines - significantly larger than prior work. Our evaluation shows that state-of-the-art algorithms V-SZZ and LLM4SZZ achieve only 33.3%-40.1% F1, confirming that using existing approaches still entails significant manual effort. VICBench enables robust evaluation of vulnerability detection approaches.