Diff vs Claims
Weight: 5 · Pro tier
Pro signal
This signal is available on Pro and Team plans. It provides deeper structural analysis of your code changes.Works on any PR
This signal runs on every PR — no test plan required. It analyzes the diff for structural and semantic issues beyond what claims verification covers.The Diff vs Claims signal uses an LLM to read the actual PR diff and compare it against the claims in the PR description. It identifies changes in the code that are not covered by any stated claim — the gap between what was changed and what was described.
How It Works
Vigil sends the full diff and the PR description to the LLM. The LLM analyzes each changed file, function, and code path, then identifies which changes are covered by at least one claim in the PR description and which are not. The LLM is prompted to be generous — only changes that genuinely lack coverage are flagged.
Penalty System
Each uncovered significant change incurs a 5-point penalty, up to a maximum of 30 points total. The score starts at 100 and decreases with each penalty:
| Uncovered changes | Score |
|---|---|
| 0 | 100 |
| 1 | 95 |
| 3 | 85 |
| 6+ | 70 (floor) |
Prompt Injection Protection
Since the LLM reads user-supplied content (the PR diff and description), Vigil applies prompt injection protections. Backtick characters in user content are escaped, and the prompt includes explicit data boundary instructions to prevent the LLM from treating diff content as instructions.
No Failure Cap
As an LLM-based signal, Diff vs Claims does not trigger the failure cap. Even if it scores poorly, it cannot limit the total confidence score to 70. Only deterministic signals (CI Bridge, Credential Scan, Test Execution, Coverage Mapper) can trigger the cap.