Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note2 min read

Automated Semantic Fault Localization in SysML v2: A Human-in-the-Loop Framework Using Knowledge-Graph Augmented LLMs

Source

arXiv

arxiv.org/abs/2606.23395v1

Read the source

Source

Plain-Language Brief

The paper presents a framework for automatically identifying and repairing semantic mistakes in SysML v2 models, which are errors that preserve syntactic validity but violate domain rules. The framework combines a fine-tuned Small Language Model (SLM) with a domain knowledge graph encoding physical compatibility rules between system elements. It demonstrates the framework using the vehicle systems domain and shows improvements in semantic fault repair from less than 3% to more than 91%.

What the Document Says

  • SysML v2 has textual syntax for compiler-based validation of model structure and language conformance.
  • Semantic mistakes that violate domain rules but are syntactically valid can cause costly integration failures.
  • The paper proposes a human-in-the-loop framework to identify and repair such errors automatically.
  • The framework combines a fine-tuned SLM with a domain knowledge graph of physical compatibility rules between system elements.
  • The knowledge graph also generates synthetic training data by introducing plausible domain violations and augments the model at inference time for repair suggestions.
  • The framework demonstrates its use in the vehicle systems domain, focusing on mechanical, electrical, fluid, and signal interfaces.
  • Two SLMs (Qwen2.5-Coder-1.5B and DeepSeek-Coder-6.7B) are fine-tuned to output unified diff patches for fault localization and repair suggestions.
  • Evaluation shows improvements in semantic fault repair from less than 3% to more than 91%, with patch-based output reducing token length by over 60%.

Key Concepts Mentioned

  • SysML v2
  • semantic mistakes
  • human-in-the-loop framework
  • Small Language Model (SLM)
  • domain knowledge graph
  • physical compatibility rules
  • vehicle systems
  • fault localization
  • repair suggestions
  • evaluation

Claims Made by the Document

  • Improving semantic fault repair from less than 3% to more than 91%

Evidence Used

  • Evaluation of 1,184 test samples shows that fine-tuning improves semantic fault repair from less than 3% to more than 91%.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

SysML v2's textual syntax enables compiler-based validation of model structure and language conformance. However, semantic mistakes that preserve syntactic validity but violate domain rules cannot be detected through compilers. These errors can propagate through the design process and surface late as costly integration failures. This paper presents a human-in-the-loop framework for identifying and repairing such errors automatically. It combines a fine-tuned Small Language Model (SLM) with a domain knowledge graph encoding physical compatibility rules between system elements. The knowledge graph also guides the generation of synthetic training data by systematically introducing plausible domain violations, and augments the model at inference time to ground repair suggestions in valid engineering constraints. We demonstrate the framework using the vehicle systems domain, where the knowledge graph captures the relationships between the mechanical, electrical, fluid, and signal interfaces. Two SLMs, Qwen2.5-Coder-1.5B and DeepSeek-Coder-6.7B, are fine-tuned to output unified diff patches that localize faults and present candidate repairs for engineer review, preserving human judgment in the design process. Evaluation of 1,184 test samples shows that fine-tuning improves semantic fault repair from less than 3% to more than 91%, with patch-based output reducing token length by over 60%. The framework offers a practical path toward AI-assisted model verification that complements existing MBSE tools.