Notes from building with agents.
Patterns that survived contact with real work. Take what’s useful; most of it applies well beyond our own tools.
All notes
7-
Normalize like a skeptic
Measuring AI-assisted development honestly took more design than expected. Score scope, never lines; classify diffs, never messages; decompose, don’t zero; and drop any era with known-missing data from headline claims.
-
Pixel statistics cannot see meaning
A character passed three experiments and a cross-experiment comparison with no face. Mechanical metrics validate geometry, scale and consistency well, and validate semantics not at all.
-
Corrective feedback should be git-diffable
When an agent misbehaves, the durable fix is a short, readable rule appended to its definition. The learned policy is text you can read, diff, blame and revert.
-
The best sub-agent output is a tool call, not a report
A helper agent burns its own context on messy discovery, then injects clean, executed tool calls into the parent’s queue. The parent’s context holds the answer, not the search noise.
-
Decomposition delivers what checkpointers simulate
When every step is an independently addressable endpoint, replaying step 30 means calling step 30. And when a multi-agent build needs resuming, the resumer should be an agent, not a replay engine.
-
Separate orchestration from the unit of inference
A flow is deliberately not a workflow engine. Control flow belongs one layer up, in real code. Fusing the two couples the thing you want to iterate on to the thing you want to be boring.
-
Catalog size measures the wrong thing
A connector library is a moat only against users who can’t generate their own tools. The axis that actually matters is time to the first working integration.