The pipeline takes a ticket and delivers finished work. It plans, writes the code, runs the tests, reviews its own output, fixes what the review finds, opens a pull request. Against a single repository it needs no supervision.
Real tickets cross repositories. An API change lands in the service that provides it, then in the client library that calls it, then in the tool that wraps that client, in that order. The pipeline stops at the repository boundary.
A stop is the pipeline halting and handing back to a person. It reports what it found, what it expected and what it needs, then does nothing further.
A stop is the pipeline declining to guess, and usually that is correct. Carrying on past a mismatch produces code that compiles, passes its own tests, and breaks the contract the next repository depends on.
The problem is frequency. Every stop interrupts a person.
The goal: reduce how often a stop is necessary, without turning a legitimate stop into a guess.
Both halves matter. Stopping less often by learning to guess would score better on the primary metric and make the pipeline worse.
A bigger context window does not solve it. Four questions have no answer today once a task crosses a repository boundary:
Four points, each a different kind of missing knowledge.
The ticket cannot become an ordered list of per-repository jobs, because the dependencies form a cycle or a repository it needs is not in the workspace. Nothing has been built, so nothing is at risk.
Either the plan needs something from a dependency that no contract states, or the ticket forces an architecture decision nobody has made. The first is a missing fact. The second is a missing human judgement.
A repository delivered, and what it provides no longer matches what the next one was told to expect. The run stops before the next repository starts, so no work is done against a stale assumption. The pipeline found the problem and named the right repository. It knows. It is not permitted to act.
The final gate checks the whole graph, including edges the step-by-step check cannot see. Those are the edges where a repository was never told what to expect and only declared afterwards what it used. Same shape as point 3, but everything is already built.
The instrument is validated and the first real measurement column is collected. What is missing is the loop those numbers are meant to judge — that design is the next piece of work.
Landed 10 Aug.
Read the workspace mode end to end, inventoried every blocker it can report.
Docker bed built. Injection suite at full strength — four active controls across three stop verdicts, plus a standing probe — and its first real measurement campaign ran 12 Aug.
32 stop sites, each classified by whether it lacks permission or information. This reshaped the rest of the plan. The campaign added one more entry. A stop can be evaded by rewriting the record it checks against, and the map now names how.
Made optional by T4. The gaps that cause stops are not the kind better retrieval fixes. If it runs, it runs narrow and is scored separately.
The main design work, now unblocked and next up. What to re-check after a change, what to carry forward, how to tell a fix from a loop. The legitimate-stop scenarios are its guardrails and must survive it.
The “before” column — clean path plus injected scenarios on the baseline model — was measured 12 Aug. The “after” waits on the T6 loop.
Only the baseline model's column is measured; other models are pending. The switching experiment now has its scenario suite and queues behind the loop design.
evidence: strong
evidence: strong
evidence: strong for engineered faults, not claimed for judgement calls
evidence: moderate: observed by accident, then used deliberately
evidence: one occurrence at one site, with a counter-instance at another; treated as a failure mode to design against, not a rate
Three repositories that depend on each other in a chain.
| Repository | Role in the chain |
|---|---|
moby/moby |
The engine. Defines the API and the Go client library everything else calls. |
docker/cli |
The docker command. Consumes the engine's client library. |
docker/compose |
docker compose. Consumes both the engine's client and the CLI. |
Three dependency edges from one ticket, with a forced order. The engine lands before the CLI, which lands before Compose.
The dependencies are real and versioned. Each repository has a substantial test suite that runs without exotic infrastructure. And a ticket's surface travels visibly, from an engine API option through a command-line flag into a Compose flag, which makes behaviour easy to observe.
Carving one repository into pieces was rejected as the only test bed. It exercises context selection, but not discovery, merge sequencing or version skew, which need real boundaries.
Three repositories. A new soft limit on container processes, added to the engine API,
exposed as a docker run flag, surfaced as a Compose flag. One detail is deliberate. For
the Compose-to-CLI edge the ticket does not state what Compose should expect, so the run
has to handle an unstated dependency.
Two repositories. A timeout option on the engine's client library, surfaced as a global CLI flag. A single edge, stated precisely.
Three more tickets exist only to seed scenarios. One names work that no repository in the workspace can own, one collides with a seeded design record, and one forces an open architectural decision. Each is written so the fault sits in the ticket's own text rather than being smuggled in by the harness.
Measurement runs live on a fresh workspace. Everything from the build-and-validate era — every ledger, every checkpoint artifact — was archived out before the campaign, so no earlier run contributes a row. The harness, not operator discipline, enforces the boundary. A run resumed from a development checkpoint refuses to archive as a measurement, so its timings and token counts can never quietly become a benchmark column.
The campaign's control run went through clean. Three repositories delivered in order, three cross-linked draft pull requests, every check and review passed on the first attempt, the integration gate passed. Zero stops, zero human interventions, 1h 35m 02s end to end.
The clean run earns its place as a control. It proves the instrument does not stop on its own — so every stop counted in the scenarios is attributable to the fault we planted.
Faults we plant on purpose, each a situation we already know should halt the run, so that “stops went down” can be measured against something real. Every scenario carries a manifest recording where the fault sits — the ground truth for scoring whether the run points at the right repository. Three of the five are situations where stopping is the correct outcome; they count inversely, and any change that reduces stops must leave them standing.