Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Specification-delta-driven data governance: an empirical study of the «spec-delta» as the unit of change in lakehouse data platforms

Source

arXiv

arxiv.org/abs/2608.19838v1

Read the source

Source

  • Source name: arXiv
  • Source kind: paper
  • Authors / organization: Pablo Ramirez Amador
  • Venue: arXiv: cs.AI
  • Published: 2026-08-20
  • Collected: 2026-08-21
  • URL: http://arxiv.org/abs/2608.19838v1
  • PDF: https://arxiv.org/pdf/2608.19838v1
  • DOI: UNKNOWN
  • Registry group: academic_apis_and_preprints
  • Evidence role: canonical_academic_index
  • Language: en
  • Country / region: UNKNOWN

Plain-Language Brief

The paper discusses the concept of Spec-Delta, which is the idea that changes in data platforms should be reviewed as units of change in requirements rather than just code changes. The paper proposes a formalized taxonomy of these changes and compares a spec-delta-driven workflow to a conventional code pull-request workflow without a delta, using various response variables like discovery to deployment time, defect density, cross-tool metric divergence, and reviewer cognitive load.

What the Document Says

  • The paper discusses Specification-Delta-Driven Development (SDD) where specifications are the primary governing artifacts in AI assisted work.
  • It mentions tools like GitHub Spec Kit and proposals such as Constitutional SDD that formalize this principle in software domain.
  • The executable data-contracts literature extends this principle to schema and quality enforcement at run time.
  • However, the treatment of specification delta in data platforms is empirically unexplored despite many data platform changes being contractual.
  • The paper formalizes the spec-delta concept and proposes a taxonomy of data platform changes according to their suitability for incremental specification.
  • It defines a controlled experiment comparing a spec-delta-driven workflow against a conventional code pull-request workflow without a delta.
  • Response variables include discovery to deployment time, density of defects reaching Silver and Gold lakehouse layers, cross-tool metric divergence, and reviewer cognitive load measured with NASA TLX.
  • The paper reserves a section for demonstration and laboratory instantiation on a real lakehouse environment.

Key Concepts Mentioned

  • Specification-Delta-Driven Development (SDD)
  • spec-delta
  • data platform changes
  • response variables
  • discovery to deployment time
  • defect density
  • cross-tool metric divergence
  • reviewer cognitive load

Claims Made by the Document

  • Spec-Delta concept should be applied in data platforms.
  • A controlled experiment comparing spec-delta-driven workflow against conventional code pull-request workflow without a delta.

Evidence Used

  • The paper presents a formalized taxonomy and a controlled experiment to support these claims

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Spec Driven Development SDD has consolidated the idea that the specification rather than the code should be the primary artefact governing AI assisted work. Tools such as GitHub Spec Kit, and proposals such as Constitutional SDD, have formalised this principle in the software domain, while the executable data-contracts literature has extended it to schema and quality enforcement at run time. Nevertheless, the treatment of the specification delta OpenSpec's core idea that every change should produce a reviewable increment of requirements as the unit of change in data platforms remains empirically unexplored, even though many data-platform changes are contractual (new datasets, service-level agreements, metric semantics, access policies) rather than purely code changes. This work formalises the spec-delta concept, proposes a taxonomy of data platform changes according to their suitability for incremental specification, and defines a controlled experiment comparing a spec-delta-driven workflow against a conventional code pull-request workflow without a delta. The response variables are discovery to deployment time, the density of defects reaching the Silver and Gold lakehouse layers, cross-tool metric divergence, and reviewer cognitive load measured with NASA TLX. The paper explicitly reserves a demonstration-and-laboratory section for instantiation on a real lakehouse environment. The contribution is not a tool but reproducible evidence and an applicability guide that helps to avoid the up front over specification antipattern.