Safe Self-Improvement Automation: Detect, Repair, Verify, Roll Back
The safest self-improvement engine is mostly a linter. It inventories components, detects drift with deterministic checks, proposes bounded repairs, verifies the result, and rolls back when proof fails. It does not give an agent open-ended permission to “make the system better.”
I run such an engine around an OpenClaw workspace. Its operational state is useful precisely because it is imperfect: the scanner finds real drift, while its own counters reveal why the improvement loop needs stronger reconciliation.
What the live system says today
I inspected the main-machine registry and append-only log on 28 July 2026:
| Signal | Observed value | Interpretation |
|---|---|---|
| Registered skills | 22 | Inventory grew beyond the original 13-skill launch snapshot |
| Missing improvement section | 1 | The detector still finds current drift |
| Total runs | 18 | The registry records repeated execution |
| Total suggestions | 150 | The scanner has produced substantial review load |
| Current summary | 0 applied; 8 flagged | This run was report-only or found no auto-fixable items |
| Lifetime applied counter | 0 | Contradicts the log, which records many successful applications |
The last line matters. A self-improvement system that cannot reconcile its registry with its findings ledger cannot accurately report success rate, rollback rate, or change burden. Detection value remains; improvement claims do not.
The control loop
manifest ──▶ scan ──▶ finding ──▶ risk classification
│
┌─────────────┴─────────────┐
▼ ▼
propose bounded auto-fix
│ │
▼ ▼
human review verify change
│
┌─────────────┴─────────────┐
▼ ▼
commit rollback
│ │
└────────▶ reconcile ◀──────┘Every arrow needs durable evidence. The next scan should verify persistence, but immediate verification belongs in the same run. Waiting until tomorrow leaves a bad mutation active for a day.
Make the inventory declarative
For each component, record:
- stable ID, type, owner, and paths;
- expected sections, files, registrations, and review interval;
- checks with severity and evidence requirements;
- allowed remediation class;
- verification and rollback procedure;
- escalation route.
Discovery can suggest new entries, but it should not silently enroll arbitrary files. A reviewed manifest defines scope; filesystem traversal only observes it.
Findings need identity and evidence
A finding is not a log line. Give it a stable ID and lifecycle:
proposed → approved → applied → verified
│ │ │
├─rejected └─expired ├─failed → rolled_back
│ └─failed → escalated
└─supersededStore run ID, check version, component ID, file digest, observed value, expected value, severity, and disposition. This makes duplicate findings idempotent and lets the engine measure whether flags lead to useful action.
Classify auto-fixes by blast radius
| Risk class | Example | Default action |
|---|---|---|
| Documentation-only | Append a reviewed section to one declared Markdown file | Auto-fix under strict preconditions |
| Reversible configuration | Add a missing non-secret registry entry | Propose or canary; verify parser and behavior |
| Behavioral code | Change retry or threshold logic | Human review and tests |
| External effect | Edit cron, deploy, message, delete, rotate credentials | Explicit approval and receipt |
| Unknown | LLM-generated refactor | Report only |
The existing engine correctly refuses to auto-tune an email threshold that was initially set to 60 without outcome data. “The value looks arbitrary” is evidence for an experiment, not evidence for a replacement value.
Preconditions prevent stale-write damage
Before an automatic edit:
- confirm the target remains inside the manifest scope;
- require a clean worktree or isolate the change;
- recompute the file digest captured during scanning;
- confirm the defect still exists;
- enforce per-run file and line limits;
- capture exact pre-change bytes for rollback.
This closes the time-of-check/time-of-use gap. If a human edits the file after the scan, the engine must rescan rather than append against stale assumptions.
Verification must prove behavior
Verification should match the repair:
- re-run the original detector;
- parse or compile the edited artifact;
- run targeted tests;
- confirm only declared files changed;
- run an integration check for wiring changes;
- record commands, exit codes, and evidence URIs.
“The text is present” verifies a documentation section. It does not verify that a heartbeat, tool, or guardrail is actually wired into the runtime.
Rollback is part of the action
Define rollback before applying the fix. For a single-file documentation edit, restoring captured bytes may be enough. For configuration, use a versioned previous value. For a deployment, require the platform's rollback handle.
If verification and rollback both fail, stop the run and escalate. Continuing to “improve” other components after losing control of one mutation multiplies uncertainty.
Reconcile the engine's own books
At run end, derive counters from the append-only findings ledger:
applied_count = count(disposition in [applied, verified])
verified_count = count(disposition == verified)
failed_count = count(disposition == failed)
rollback_count = count(disposition == rolled_back)
assert verified_count <= applied_count
assert every failed application has rollback or escalation evidence
assert summary counters equal ledger countsThis directly addresses the current registry/log contradiction. A mutable lifetime counter should be a cache, never the source of truth.
Metrics that resist vanity
| Metric | Question answered |
|---|---|
| Finding precision | How many reviewed findings were real? |
| Action rate | How many findings led to an approved change? |
| Verification success | How many applied changes proved their intended effect? |
| Rollback rate | How often did the engine need to recover? |
| Recurrence rate | Did the same defect return? |
| Time to disposition | Are flags becoming an ignored backlog? |
| Change-induced incidents | Did the improvement engine cause harm? |
“Suggestions generated” measures output volume, not improvement.