Designing a 90% Automated Product Pipeline—What Must Stay Human

I need to correct the original version of this article: I had not measured 90% automation.

DreamForge is a detailed product-to-code architecture, not yet a validated end-to-end product. Its design is complete and its first pilot is specified, but the project record is still at “Awaiting Phase 0 Validation.” There is no defensible before-and-after cycle time, failure rate, or production cost yet.

The useful question is therefore not “How did I automate 90%?” It is: What would have to be true before 90% automation was safe and measurable?

The root problem is translation loss

DreamForge began with a familiar gap: product plans lived in research notes, while implementation began in a coding session. Every handoff required a human to reconstruct intent.

Five Whys exposed the system problem:

1. Why was delivery slow? Plans had to be manually translated into coding tasks.
2. Why did translation require a human? Product documents were written for readers, not execution.
3. Why could an agent not simply read the document? Acceptance criteria, dependencies, constraints, and decision authority were implicit.
4. Why did adding more agents not solve it? More handoffs multiplied ambiguity and state loss.
5. Why design DreamForge? The pipeline needed a typed contract, durable state, explicit ownership, and bounded autonomy.

The goal is not to remove the founder from product development. It is to stop spending founder attention on loss-prone translation.

What exists today

Artifact	Status	What it proves
Product-to-code architecture	Complete design	Roles, state, messages, circuit breakers, and recovery paths are specified
Failure-mode review	21 scenarios documented	The happy path has been challenged on reliability, security, cost, and human experience
Structured intermediate representation	Contract designed	Features can carry IDs, acceptance criteria, dependencies, and architecture references
EchoWeave pilot	Candidate only	A complex test product is defined, but no completed end-to-end run is recorded
Measured 90% automation	Not established	The percentage remains a target, not a result

The proposed pipeline

Human: problem, desired outcome, constraints
                    │
                    ▼
Planner: research → architecture → typed spec
                    │
              feasibility gate
                    ▼
Implementer: code → tests → checkpoints → PR
                    │
                    ▼
Auditor: spec diff → risk checks → compliance report
                    │
                    ▼
Human: review → merge → production decision
                    │
                    ▼
Post-implementation review updates the spec template

The repository is the durable system of record. The spec, decisions, code, tests, and checkpoints are versioned. Chat is used for notification and human clarification, not as the message bus or the memory layer.

This was a concrete rejected design decision. The initial proposal used Discord tags for agent communication and Jira for tracking. The critique rejected that combination as too many handoffs and too little replayability. The revised design makes typed files and Git the contract, with chat off the critical path.

What the agent can own

Agents are strongest where work is explicit, observable, reversible, and cheaply testable:

• turning an approved problem into a first structured specification;
• mapping feature dependencies and generating task order;
• implementing bounded changes in a sandbox;
• running tests and committing recovery checkpoints;
• comparing code with acceptance criteria;
• tracking cost, retries, clarification count, and state;
• preparing a pull request and an evidence report.

None of those tasks requires pretending that a model owns the product outcome.

What must stay human

Decision	Why it stays human	Agent contribution
Which problem deserves investment	It allocates scarce time and encodes strategy	Assemble evidence and expose assumptions
What user outcome is acceptable	Tests can satisfy a wrong specification	Draft measurable acceptance criteria
Breaking API, security, or data decisions	Blast radius and obligations exceed a local code change	Present options, threats, and rollback costs
Large scope or cost changes	The pipeline should not silently redefine the investment	Pause and provide the variance
Merge and production deployment	A passing suite is evidence, not authorization	Prepare the PR, release plan, and rollback checks

“Zero human intervention” is only a sensible pilot metric from approved spec to reviewable pull request. It must not mean unattended production deployment.

A practical delegation test

Score a stage on five questions:

1. Impact: How much damage can a wrong decision cause?
2. Reversibility: Can the result be rolled back without data or trust loss?
3. Observability: Will tests and telemetry reveal failure quickly?
4. Ambiguity: Is the desired outcome explicit enough to encode?
5. Recovery cost: How expensive is human correction?

Delegate by default only when impact and recovery cost are low, reversibility and observability are high, and ambiguity is bounded. Otherwise, require approval before execution or keep the decision human-owned.

Circuit breakers turn judgment into policy

Human oversight should not mean approving every line. It should mean defining conditions that force a pause:

Trigger in the design	Required action
Feature cost exceeds €10	Pause and request an explicit override
Idea cost exceeds €50	Pause the pipeline and review scope
Scope changes by more than 20%	Return to the product owner
Breaking API or security-boundary change	Require architecture and risk approval
Planner–implementer–auditor loop exceeds three cycles	Stop automation and diagnose the specification
Auditor score falls below the pilot threshold	Do not present the change as ready to merge

These thresholds are design hypotheses. Phase 0 must determine whether they are useful, too permissive, or too noisy.

How to measure the 90% target

Counting generated files is not automation. Measure eligible delivery steps completed without human intervention:

automation coverage =
  unattended eligible steps completed successfully
  -------------------------------------------------
  all eligible steps attempted

Report separately:
  human touch time
  elapsed cycle time
  rework time
  escaped defects
  clarification count
  agent and infrastructure cost

Do not include problem selection, production authorization, or other deliberately human-owned decisions in the denominator. A pipeline should not be penalized for respecting its safety boundary.

The validation plan

DreamForge’s documented Phase 0 is intentionally manual for one or two ideas. A human plays orchestrator and records spec quality, clarification frequency, and implementation accuracy. The exit criteria are fewer than three clarifications per feature and more than 80% spec-to-build accuracy.

The later EchoWeave pilot proposes:

• a full run from idea to pull request without intervention;
• fewer than five clarifications in total;
• less than $15 of model cost;
• a compliance score of at least 0.90;
• simulated tests for all 21 documented failure modes.

Those are acceptance targets, not observed results. A credible case study should publish the baseline and actual outcome after the pilot, including the failures.

The real lesson

The valuable artifact in an autonomous delivery system is not the model roster. It is the boundary: a typed spec, explicit state transitions, durable decisions, circuit breakers, evidence at every handoff, and a human with clear authority.

For the contract-first argument, read Stop Building Agent Soup. For the delivery tradeoff, see Quality vs. Speed in the Agent Era. For a working cross-agent handoff with measured tasks, see the orchestrator-to-implementer case study.

You can also download the human-vs-agent responsibility matrix and adapt its gates to your own pipeline.