Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Tuning the Stochastic Machine: A Systems Engineer's Operating Model for Human-AI Engineering

Source

arXiv

arxiv.org/abs/2608.19125v1

Read the source

Source

  • Source name: arXiv
  • Source kind: paper
  • Authors / organization: George Andrikopoulos
  • Venue: arXiv: cs.AI
  • Published: 2026-08-19
  • Collected: 2026-08-20
  • URL: http://arxiv.org/abs/2608.19125v1
  • PDF: https://arxiv.org/pdf/2608.19125v1
  • DOI: UNKNOWN
  • Registry group: academic_apis_and_preprints
  • Evidence role: canonical_academic_index
  • Language: en
  • Country / region: UNKNOWN

Plain-Language Brief

The author, a systems engineer, argues that issues with Large Language Models (LLMs) are more about operational procedures than tooling. They propose an operating model for human-AI engineering based on seven principles, highlighting the importance of persisting corrections and monitoring recurring errors. The text also includes three real-life examples and suggests a lab study to test this model.

What the Document Says

  • The author argues that the issue of corrections dying with sessions in LLMs is an operations problem rather than a tooling problem.
  • Mechanisms for persisting corrections exist and are being shipped, but the discipline for governing them is lacking.
  • The author maps the LLM stack onto machines operated by their profession (frozen silicon, firmware, etc.)
  • The mapping fails in stochastic generation, configuration that binds probabilistically, and lack of a general-purpose retirement (verification) stage by default.
  • The author proposes a seven-principle operating discipline with an error loop at its core based on these failures.
  • Three real-life examples are provided to illustrate the mechanism, including a control that became the exact harm it was built to prevent.
  • The text suggests a lab study to test the proposed model.

Key Concepts Mentioned

  • Large Language Models (LLMs)
  • Operations problem
  • Tooling problem
  • Machines
  • Stochastic generation
  • Configuration
  • Provenance
  • Recurrence monitoring
  • Counter-metrics
  • Retirement of stale rules
  • Error loop
  • Seven-principle operating discipline

Claims Made by the Document

  • Issues with LLMs are more about operations than tooling.
  • Persisting corrections and governing them is crucial but currently lacking.
  • The author's profession operates machines similar to the LLM stack.
  • The mapping between the two fails in specific areas.
  • The author proposes a seven-principle operating discipline based on these failures.
  • Three real-life examples are provided to illustrate the proposed model.
  • A lab study is suggested to test the model.

Evidence Used

  • When an expert corrects an LLM assistant's error, the correction usually dies with the session...

Methods / Approach

  • Mapping the LLM stack onto machines operated by the author's profession
  • Proposing a seven-principle operating discipline

Limitations Stated by the Authors

  • The author's perspective is limited to their own practice and experiences.

Limitations Noted by the Agent

Agent interpretation — not from the source.

  • The text does not provide details on the specific lab study suggested.

Open Questions

  • What are the specific seven principles of the proposed operating discipline?
  • How effective would this operating discipline be in addressing issues with LLMs?

Notes for Future Q&A

May include interpretation. Labeled where it does.

  • The text does not explicitly state whether the seven principles have been fully developed or tested.
  • The effectiveness of the proposed operating discipline is not evaluated within the provided text.

Raw Abstract / Excerpt

When an expert corrects an LLM assistant's error, the correction usually dies with the session, and the error class returns. I argue this is an operations problem, not a tooling problem: mechanisms for persisting corrections exist and are shipping, but the discipline for governing them -- versioning with provenance, recurrence monitoring, counter-metrics, retirement of stale rules -- does not. Writing as a systems engineer of thirty years, I map the LLM stack onto the machines my profession already operates (frozen silicon, firmware, loadable modules, persistent configuration, volatile memory), identify where the mapping fails (stochastic generation, configuration that binds only probabilistically, no general-purpose retirement (verification) stage by default), and derive from the failures a seven-principle operating discipline with an error loop at its core. Three cases from my own practice illustrate the mechanism, among them a control that silently became the exact harm it was built to prevent. I close with the measurement framework this view implies and the lab study required to test it.