Close The Running Loop

A practical guide to Peter Steinberger's "Close the Loop" methodology for autonomous AI agent coding — with a real experiment proving it works.

The idea is simple: give your coding agent the ability to verify its own work.

Peter Steinberger (creator of OpenClaw/Clawdbot, founder of PSPDFKit) ships code he doesn't read. He runs 3-8 AI coding agents in parallel and merged 600 commits in a single day. The secret? Every agent writes tests, runs them, fixes failures, and only reports back when everything passes. No human in the loop for verification.

"Code works well with AI because it's verifiable. You can compile it, run it, test it. That's the loop. You have to close the loop." — Peter Steinberger

Start at the bottom. Each layer adds realism — and cost.

         ┌──────────┐
         │  Docker   │  Full app in container, cold-start
         │  Tests    │  Slowest, most realistic
         ├──────────┤
         │   Live    │  Real API calls (Stripe, Anthropic, etc.)
         │  Tests    │  Costs money, flaky (network/rate limits)
         ├──────────┤
         │   E2E     │  API routes, webhooks, request/response
         │  Tests    │  Minutes, catches integration bugs
         ├──────────┤
         │   Unit    │  Pure logic, validation, security
         │  Tests    │  Seconds, catches 80% of bugs
         └──────────┘

Where to start: Unit tests. They give you 80% of the value at <1% of the cost. Peter's OpenClaw has 1,376 test files and unit tests are the foundation.

When to add E2E: When you have API routes that handle auth, data mutation, or external integrations. Tests like "does POST /api/tasks return 201?" catch a different class of bugs than unit tests.

When to add Live: Only when you integrate with third-party APIs and need to verify they haven't changed their format/behavior. These are expensive and flaky — run them manually or before releases, not on every commit.

When to add Docker: When you deploy to production and want to verify the entire app starts cleanly from scratch. Good for CI/CD pipelines.

1. Colocated tests — Every source file gets a .test.ts right next to it
2. Close the loop — After writing code, run tests. If they fail, fix. Don't ask the human.
3. Full gate before push — build + lint + test must all pass
4. Anti-redundancy — Search for existing helpers before creating new ones

We analyzed the OpenClaw GitHub repo:

• 1,376 test files across the codebase
• 563 source files with colocated .test.ts files
• Largest test files are 45-65KB — real tests, not stubs
• Recent commits consistently pair source changes with test changes

Here's a ready-to-use prompt for your coding agent (adapt the stack):

You are working on my project which has a [YOUR STACK].
We currently have zero tests.

I want to implement Peter Steinberger's "Close the Loop" methodology, where the agent
(you) can test and verify your own work autonomously via CLI commands, without needing
a human to check every change.

TESTING STRUCTURE
- Write a test file for every file you create or modify
- Colocate tests next to the source file they test
- Name them: source.test.ts next to source.ts
- Start with unit tests only

CLOSE THE LOOP
- After writing any code, run the tests via CLI before considering the task done
- If tests fail, fix the code and run again — do not ask me to check
- Only report back when tests are passing

COMMANDS TO RUN
- After writing code: run tests, linter, type checker
- Before any PR: run full gate (build + lint + test)
- Never push failing code

ANTI-REDUNDANCY
- Before creating any helper or utility, search for existing ones first
- If a function already exists, import it — do not duplicate it
- Extract shared test fixtures into test-helpers files when used in multiple tests

We didn't just implement this — we tested whether the tests actually catch bugs.

📄 Full Experiment Report →

• Project: Entity (Next.js + TypeScript monorepo)
• Test runner: Vitest
• Modules tested: classify.ts (file classification) and security.ts (path security)
• Total tests written: 64 across 3 files
• Test execution time: <500ms

1. Wrote comprehensive tests for 2 modules (64 tests total)
2. Verified all tests pass (baseline)
3. Introduced 6 deliberate bugs — mix of logic inversions, missing cases, off-by-ones, and a critical security bypass
4. Ran tests to see how many bugs get caught

The tests caught:

• Both logic inversions
• A missing code path
• A critical security vulnerability (path traversal bypass)
• A missing configuration entry

The one it missed: a subtle off-by-one where the test wasn't testing the boundary value. Lesson: test boundary values, not just happy paths.

1. Speed matters. If tests take >1 minute, the loop is too slow. Our tests run in <500ms.
2. Quality > Quantity. 64 tests missed one bug because we didn't test the boundary value. Edge cases are the critical differentiator.
3. Colocated = discovered. When tests sit next to source files, agents naturally find and run them.
4. Security bugs get caught. The most important bug in our experiment (path traversal bypass) was caught immediately.
5. It's not magic. Close the loop works because code is verifiable. The agent can objectively check if its work is correct.

This methodology was researched after a conversation between Henry Mascot and Kinan Zayat, who reverse-engineered Peter Steinberger's approach by studying the OpenClaw codebase. Kinan identified the 4-file pattern and adapted it for his own Next.js stack.

• Peter Steinberger's blog: "Just Talk To It"
• The Pragmatic Engineer Podcast interview
• OpenClaw GitHub
• Peter's AGENTS.md gist

MIT — use this however you want.