AgentsIntelligence Snack

Long Agent Runs Depend on Reliable Feedback

Tests, specifications and human review determine how long a coding agent can keep producing useful software on its own.

Developed from a conversation between Andy David, Pete Winn and dpc

From Episode 64: Vibe Coding with Power Tools

Coding agents can produce large amounts of software, so the useful length of an autonomous run depends on the signals available to keep the generated code aligned with its requirements. As generation becomes faster, the quality of those evaluation signals increasingly sets the limit.

Compilers, tests and stable APIs provide mechanical feedback that acts like a mould around a replacement implementation. Repeated checks can confirm that new internal code returns the same API responses and passes the same tests, giving an agent clear boundaries for a rewrite. Business applications are harder to steer because many requirements live in button placement, approval sequences, human workflows and expectations that have never been written down.

An application can preserve its data and become harder to use because a working button can appear in the wrong place or an approval flow can violate an unstated operational expectation. The agent receives no automatic signal that either mistake has occurred. Longer runs therefore need more requirements expressed through tests, specifications and other machine-readable constraints, with human review covering usability and operational knowledge.

Get Intelligence Snacks in your inbox.

Quickly digest the big ideas emerging from the world of AI, delivered each week.