Five Infrastructure Layers Every Production AI Agent Needs

A production agent is not a model with tools. It is a controlled execution system in which a probabilistic planner is one replaceable component.

This reference architecture grew out of real failures in my own system, but it is not another incident narrative. It defines the five infrastructure layers I now expect to see before an agent can take consequential action.

The reference architecture

Human, service, schedule, or event
                 │
                 ▼
┌─────────────────────────────────────────────┐
│ 1. Interface & Transport                    │
│ authenticate · normalize · deduplicate      │
└──────────────────────┬──────────────────────┘
                       ▼
┌─────────────────────────────────────────────┐
│ 2. Identity, Policy & Authority             │
│ allow/deny · approval · scope · budget      │
└──────────────────────┬──────────────────────┘
                       ▼
┌─────────────────────────────────────────────┐
│ 3. Orchestration, State & Model Routing     │
│ plan · journal · execute · resume           │
└──────────────────────┬──────────────────────┘
                       ▼
┌─────────────────────────────────────────────┐
│ 4. Verification, Recovery & Change Control  │
│ invariants · sandbox · rollback · receipt   │
└──────────────────────┬──────────────────────┘
                       ▼
                External system

Every layer ──events──▶ 5. Observability,
                         Evaluation & Learning

The arrows are contracts, not prompt text. A layer may be a library in a small deployment or a separate service at scale. What matters is that its responsibility and evidence are explicit.

Layer 1: interface and transport

This layer converts email, chat, API calls, schedules, queues, and human requests into one trusted envelope. It should not decide what the agent is allowed to do.

Minimum input contract: request ID, source, authenticated principal, timestamp, payload schema version, idempotency key, data classification, and reply channel.

Failure mode	Control	Readiness evidence
Duplicate event triggers two actions	Idempotency store and replay-safe handler	Same event replayed without a second side effect
Provider outage stops the workflow	Adapter boundary, durable queue, degraded mode	Failover or queued recovery exercised
Untrusted content becomes instruction	Source labels, schema validation, content/instruction separation	Injection test corpus rejected or contained
Credentials leak into model context	Credential broker outside prompt context	Trace proves secrets never enter messages

The output is a normalized request or a typed rejection. “The model can parse it” is not a transport contract.

Layer 2: identity, policy, and authority

Authentication says who initiated the request. Authorization says what that principal and this agent may do. Policy adds context: risk class, data boundary, budget, environment, time, and required approval.

Decision contract: allow, deny, or require approval; permitted tools and resources; financial and rate limits; expiration; reason; policy version; approving principal.

Failure mode	Control	Readiness evidence
Agent expands its own mandate	Deny-by-default capabilities with scoped grants	Privilege-escalation tests fail closed
Prompt injection requests a secret or new tool	Policy enforcement outside the model	Adversarial request cannot alter policy decision
Approval is vague or reusable forever	Action-specific, expiring approval token	Changed payload invalidates approval
Budget runs away	Per-run and per-period hard limits	Limit test stops execution with a receipt

This layer is the architectural response to excessive agency. OWASP's Top 10 for Agentic Applications is a useful threat-model companion, but a checklist cannot replace enforcement in the execution path.

Layer 3: orchestration, state, and model routing

The orchestrator turns an authorized intent into a bounded plan, executes tools, records state, and resumes safely after interruption. The model proposes; the state machine owns progression.

Execution contract: run ID, immutable intent, plan version, current step, allowed transitions, tool inputs and results, retry count, deadline, model route, and terminal outcome.

Failure mode	Control	Readiness evidence
Retry repeats a non-idempotent mutation	Effect journal and idempotency key per step	Crash-after-write test resumes without duplication
Two actors edit the same state	Lease, version check, or serialized writer	Concurrent-update test detects conflict
Cheap model violates a critical constraint	Route by risk and capability; validate independently	Evaluation set justifies each route
Context loss changes the goal	Immutable intent plus compact structured state	Resume test preserves constraints across sessions

Model routing belongs here because it is an execution decision. Select a route using task risk, required capability, latency, privacy, and measured performance—not brand preference.

Layer 4: verification, recovery, and change control

Verification is not a final “looks good” prompt. It is a set of deterministic and human gates around effects. The checks must match the artifact: parser and tests for code, schema and constraints for data, render inspection for visual output, approval for irreversible external action.

Verdict contract: artifact or proposed action hash, checks executed, invariant results, reviewer identity where required, decision, rollback reference, and effect receipt.

Failure mode	Control	Readiness evidence
Syntactically valid output is operationally wrong	Domain invariants and scenario tests	Known-bad fixture is rejected
Self-modification corrupts executable files	Typed mutation path, diff, test, atomic replace	Invalid mutation leaves original intact
Rollback exists only on paper	Versioned artifact and rehearsed restoration	Timed rollback drill meets objective
Human approves a different action	Approval bound to exact action hash	Any mutation forces reapproval

High-risk changes should use preview → verify → approve → apply. The verifier must not share the same blind spot as the generator.

Layer 5: observability, evaluation, and learning

Logs are not observability if they cannot answer who asked, what was authorized, which model and tools acted, what changed, what it cost, and whether the result satisfied the objective.

Event contract: trace ID, timestamp, actor, policy and plan versions, step, tool, model, sanitized input/output references, latency, cost, result, error class, approval, and effect receipt.

Failure mode	Control	Readiness evidence
Failure is discovered by a user	Health checks, synthetic transactions, SLO alerts	Injected fault pages the correct owner
Trace logs leak sensitive data	Structured redaction and retention policy	Secret-seeded test leaves no secret in telemetry
“Learning” repeats a bad conclusion	Evidence-linked proposal and review gate	Unverified lesson cannot modify policy or code
Reliability cannot be compared	Versioned evaluation set and stable metrics	Release report shows baseline and regression

The NIST Generative AI Profile provides a broader governance and risk-management frame. This layer supplies the operational evidence needed to make that work concrete.

The end-to-end contract

For a consequential action, one trace should let an operator reconstruct this chain:

1. The transport authenticated and normalized request R.
2. Policy version P authorized exact scope S.
3. Plan version V executed steps with journal J.
4. Verification suite T passed artifact or action hash H.
5. Effect receipt E records what the external system accepted.
6. Telemetry links R → P → V → J → T → H → E without exposing secrets.

If one link is missing, the system may still be useful, but it is not independently auditable.

Production readiness gates

Gate	Minimum evidence before production
Interface	Replay, malformed input, provider outage, and injection tests
Authority	Denied privilege escalation, exact approval binding, budget-stop test
Execution	Crash/resume, duplicate suppression, conflict, and deadline tests
Verification	Known-bad rejection, rollback drill, irreversible-action approval
Operations	SLOs, owner, alert route, runbook, redaction test, release evaluation

A demo proves the happy path once. Production readiness proves bounded failure and recovery repeatedly.

Download the readiness assessment

The assessment turns every control above into an evidence field with an owner, status, last-tested date, and remediation gap.

Download the production agent readiness assessment

For the incidents that produced this model, read Five Failures From Three Months Building a Personal AI Agent. For measurement, continue with the agent reliability SLO framework and the memory architecture.