← all posts

Your AI Productivity Metric Is a Lie. Here's a Better One.

ai-assisted engineering — part 1 of 4

Everyone counts AI-generated lines of code. Nobody counts the fix commits that follow.


On January 23, I shipped a Stripe Connect integration. The AI wrote the whole thing: account onboarding, payment intents, platform fees, webhook processing. The commit landed clean. It looked done.

Three days later, four fix commits in 32 minutes. Hardcoded localhost URLs that broke in production. A Payment Element that unmounted during confirmation. A missing test mode banner. Error handling gaps that only surfaced under real Stripe API conditions. The feature "worked" on January 23. It actually worked on January 26.

This kept happening. A feature lands, it looks complete, and then a real server or a real API finds what the AI missed. Fixes follow within minutes. Sometimes seconds.

3 days feature lands 4 fixes · 32 min jan 23 jan 26 "looks done" "actually done"

I tracked it across 7 projects, 200+ commits, and ~45,000 lines of code. The data is more interesting than the headlines.

the metric everyone uses is wrong

David Crawshaw, building exe.dev, says AI writes 90% of his code. The YC CEO's CLAUDE.md reportedly ships 4,000+ line features in an hour. Peter Steinberger runs 3-8 agents in parallel on a 300K LOC TypeScript project where agents write 100% of the code.

These are real practitioners. I believe their numbers. But the numbers measure the wrong thing.

They count what the AI generates: lines written, features committed, co-author tags. They don't count what happens next.

what the commit history actually shows

I used conventional commits (feat:, fix:, docs:) across all my projects, which means the git log is a machine-readable record of what was a feature and what was a fix.

Production infrastructure (Terraform + Docker + Hetzner) 21 commits over 14 days. 12 of them (57%) are fixes. Fix-to-feature ratio: 2.4:1. Every single feature commit is followed by 1-3 fix commits:

Feature Fixes Time to first fix
Init infra repo 3 fixes (Terraform backend, cloud-init, deploy script) 7 minutes
Add Blackbox Exporter 1 fix (network connectivity) 1 minute
Add Grafana provisioning 1 fix (recursive copy in deploy.sh) 2 minutes
Add PostgreSQL monitoring 3 fixes (image registry, connection string, dashboard) 1 minute
Zero-downtime deployments 3 fixes (project name, no-deps flag, IPv4) 9 minutes

Full-stack SaaS (FastAPI + React + Stripe) 69 commits over 24 days. 16 fixes for 25 features, a 0.64:1 ratio. Seven times in the history, a fix commit directly follows a feature commit within minutes. The Stripe integration alone: 4 fixes in 32 minutes across a 3-day gap. A database migration cascade: 4 fixes in 20 minutes. My most productive day was 13 commits. Seven features, six fixes.

OpenClaw infrastructure (4 repos) 49 commits across 4 repositories. 15 feature commits, 15 fix commits. A perfect 1:1. The config repo shows feat-to-fix gaps as short as 1 minute, with a Python dependency missing from a Docker image caught 43 seconds after the feature landed.

The fix-to-feature ratio is a better productivity metric than lines of code. If you only count the feature commits, you're lying about your velocity.

feat fix infrastructure 5 12 2.4 : 1 saas 25 16 0.64 : 1 openclaw 15 15 1 : 1 fix : feat

why this happens

AI writes structurally correct code that's environmentally naive.

It doesn't know that localhost resolves to IPv6 on your server. It doesn't know your PostgreSQL password contains base64 characters that break URL parsing. It doesn't know your Docker containers were created under a different project name. It doesn't know the node:22-bookworm-slim base image doesn't include Python.

Every one of those is a real bug from my repos. Every one was caught by me deploying to a real server and watching it fail. The AI never caught a single one of its own bugs proactively. Not once across 200+ commits.

This isn't a quality problem. The code is well-structured. The Terraform is valid. The Docker Compose parses. The shell scripts have set -euo pipefail and consistent error handling. Everything passes static checks. It fails on contact with reality, the runtime environment it's never seen.

Simon Willison calls this "vibe engineering" and frames it well: you're "managing a growing army of weird digital interns who will absolutely cheat if you give them a chance." The interns write fast, but you review everything.

the 90% number and the 57% number are both true

Crawshaw's 90% and my 57% fix rate don't contradict each other. They measure different things.

Crawshaw also reports his code reading-to-writing ratio shifted from 50-50 to 95-5. He reads almost everything now, writes almost nothing. That shift tells you something: when AI generates the code, the human work moves to reviewing and verifying. The output is faster, but the oversight is constant.

An HBR study of 200 employees at a US tech company confirms this. AI doesn't reduce work, it intensifies it. Workers manage multiple threads in parallel and feel a sense of momentum, but the reality is constant context-switching and frequent output checking. Willison, quoted in the piece, says "after just an hour or two my mental energy for the day feels almost entirely depleted."

My commit timestamps tell the same story. The productive days aren't relaxed. They're 13-commit stretches with fixes landing 1-9 minutes after features. The AI generates fast. The human validates faster. It's intense.

Everyone's measuring "lines of code generated by AI." That's like measuring a factory's output by counting everything that comes off the line, including the defects sent back for rework.

when the ratio compresses

The fix-to-feature ratio isn't destiny. It improved across my projects, and the reason was always the same.

In my booking system, the first 13 days (before I wrote a project instruction file) produced migration cascades, non-conventional commits, and infrastructure that got built then thrown away. From day 14 onward, after adding a 189-line CLAUDE.md that documented conventions, gotchas, and architecture, commit quality improved immediately. Conventional commit format across the board. Fewer fix-after-feature patterns. Features landing closer to correct on the first try.

Mitchell Hashimoto describes the same thing: "anytime an agent makes a mistake, engineer a solution so it never happens again." His Ghostty AGENTS.md has every line mapping to a fixed bad behavior. StrongDM went further and moved from boolean "tests green" to probabilistic "satisfaction", measuring what fraction of AI trajectories actually satisfy the user.

The fix-to-feature ratio compresses when you front-load judgment. Instruction files that encode your conventions. Specs detailed enough that "correct-shaped wrong code" becomes just correct code. Pre-commit hooks that catch the classes of errors AI is most likely to make. Validation layers between generation and deployment.

The ratio never reaches zero. AI will always miss things that only surface on real hardware, with real data, under real conditions. But you can move from 2.4:1 to something closer to 1:3. One fix for every three features. That's where the real productivity gain lives.

what to measure instead

If you're an engineering leader evaluating AI coding tools, stop measuring lines generated. Measure these:

Fix-to-feature ratio. Count your feat: and fix: commits. If you're not using conventional commits, start. They make AI output auditable. A ratio above 1:1 means the AI is generating more rework than value. Below 0.5:1, you're in a good place.

Time-to-correct. How long between a feature commit and its first fix? Minutes means the feedback loop is tight, the human is catching issues fast. Days means bugs are reaching production. My Stripe integration had a 3-day gap. That's too long.

Rework cycles per feature. The Stripe integration needed 4 fix commits. PostgreSQL monitoring needed 3. Zero-downtime deployments needed 3. Track the average. If it's climbing, your AI context is degrading and the AI is losing institutional knowledge between sessions.

Before/after on instruction files. If you introduce a CLAUDE.md, AGENTS.md, or .cursor/rules, measure the fix-to-feature ratio before and after. In my data, the difference was immediate and persistent.

the honest number

AI-assisted development is faster than manual development. Even with the fix cycles, even with the debugging, even with the rework. Building production infrastructure in 14 days that would have taken a month is real. Shipping a full-stack SaaS with payments in 24 days is real.

But it's not 10x faster. It's maybe 2-3x, with the human spending their time differently. You write less and review more. The bottleneck moves from "can I write this code?" to "can I catch the mistakes fast enough?"

That's a meaningful shift. It's worth adopting AI tools for. But it requires an honest accounting of what the AI actually costs, not just in tokens, but in fix commits, debugging sessions, and the cognitive load of constant verification.


The fix-to-feature ratio isn't something you accept. It's something you compress.

Across my projects, one artifact consistently made the difference. A file with zero application code that changed how the AI understood my repos, my conventions, and my constraints. It cut the rework. It killed the repeated mistakes. It turned vague AI output into precise AI output.

That file, and why it might be the most important thing in your repository, is the subject of part 2.


Part 1 of a series on AI-assisted engineering, based on 7 project retrospectives covering 200+ commits. Part 2: "The Most Important File in Your Repository" explores the instruction file pattern that compresses the fix-to-feature ratio. Part 3: "The AI Doesn't Learn. The Prompt Evolves." goes deeper on prompt evolution. Part 4: "8 Tools in 12 Days" covers the memory layer.

Built in Porto. Data from real repos, real commits, real production servers.