Skip to content
← Back to blog

The Gap Between What a PR Says and What It Does

·4 min read

Your team merges 50 PRs a week. How many did someone actually read line by line? The answer is uncomfortable. And the gap between what PRs claim and what the code actually does is where bugs reach production.

The anatomy of a missed change

A PR says "Fix login timeout." The diff shows a timeout change in the auth service. Looks good, approved. But buried on line 247 of a 300-line diff, there's also a change to the session duration from 24 hours to 7 days. Nobody mentioned it. Nobody caught it.

This isn't malicious. The developer fixed the timeout and, while they were in the file, adjusted the session duration too. They forgot to mention it in the description. The reviewer saw "Fix login timeout" and focused on the timeout logic. Both humans did their job — and a change shipped undocumented.

It gets worse with scale

Small teams with 5 PRs a week can catch these manually. At 50 PRs a week, nobody reads every line. At 200 PRs — common for teams using AI coding agents — the math is impossible.

Every PR that ships with undocumented changes is a future debugging session. "When did the session duration change?" "Why is this new dependency in our lock file?" "Who added this environment variable?" The answers are buried in PR #847 from three weeks ago.

What undocumented changes look like

We've analyzed thousands of PRs. The most common undocumented changes fall into predictable categories:

New dependencies
PR description talks about a feature. The diff adds a new npm package. No mention of why it's needed or what it does.
Caught by: Undocumented Changes signal
Environment variables
A new API_TIMEOUT variable appears in the code. The PR says "improve API handling" but doesn't mention the new config requirement.
Caught by: Undocumented Changes signal
Schema changes
A database migration adds a nullable column. The PR is about a UI feature. The schema change enables the feature but isn't documented.
Caught by: Undocumented Changes signal
Behavioral changes
Default values change. Error handling switches from throw to silent return. Retry logic gets removed. All legal code changes, none mentioned.
Caught by: Claims Verifier + Undocumented Changes

Closing the gap

The fix isn't "write better PR descriptions" — humans will always forget things, and AI agents will always be confidently incomplete. The fix is automated verification.

Vigil reads every PR description, extracts the claims, verifies each against the diff, and surfaces everything the description missed. It runs on every PR automatically — no config, no manual step, no human memory required.

The gap between what a PR says and what it does doesn't have to be a mystery. It can be a report.