How It Works

From PR webhook to confidence score — what happens when you open a pull request.

The Pipeline

Vigil runs on every pull request — no configuration or test plan required. The entire process typically completes in under 60 seconds.

pipeline
PR opened/updated
    │
    ▼
Webhook received
    │
    ▼
Fetch diff from GitHub API
    │
    ▼
Run 6 verification signals in parallel
    │
    ▼
Calculate confidence score (0-100 weighted average)
    │
    ▼
Post results (PR comment + GitHub Check Run)
    │
    ▼
Send inline review comments (Pro only)

Step 1: Receive

When a PR is opened, updated, or reopened, GitHub sends a webhook to Vigil. Vigil reads the PR title, description, and diff via the GitHub API. No code is cloned or executed — all analysis is performed against the diff data.

Step 2: Analyze

Six independent verification signals run in parallel. Each signal examines a different aspect of the PR and produces a score from 0 to 100, a pass/fail status, and detailed evidence explaining its findings.

SignalWeightTierWhat it checks
Claims Verifier30FreeLLM verifies PR description claims against the actual diff
Undocumented Changes25FreeDetects meaningful changes not mentioned in the PR description
Credential Scan20FreeDetects leaked secrets, API keys, and tokens in the diff
Coverage Mapper10FreeChecks that changed files have corresponding test files
Contract Checker10ProLLM compares API response shapes vs frontend interfaces
Diff Analyzer5ProLLM analyzes diff for structural and semantic issues

On the Free tier, only the four Trust Verification signals run. Pro users get all six signals for deeper analysis. See the Signals page for detailed documentation on each signal.

Step 3: Score

The confidence score is a weighted average of all signal scores. Each signal contributes proportionally to its weight. The final score falls into one of three tiers:

  • 80-100 — Safe to merge. All critical checks pass, high confidence.
  • 50-79 — Review recommended. Some signals flagged issues worth investigating.
  • 0-49 — Caution. Significant issues detected, manual review strongly advised.

A failure cap applies: if any deterministic signal (Credential Scan, Coverage Mapper) fails critically, the score is capped at 70 regardless of other results. See Confidence Score for the full scoring methodology.

Step 4: Report

Vigil posts results in two places:

  • PR comment — A detailed breakdown with the confidence score, signal table, and actionable recommendations. Comments are idempotent: Vigil updates the same comment on subsequent runs instead of creating duplicates.
  • GitHub Check Run — A check with a conclusion of success, neutral, or failure. Use this with branch protection rules to gate merges on the confidence score.

Pro users also receive inline review comments posted directly on relevant diff lines, making findings easy to act on without scrolling through the full report.

If something looks wrong, you can re-run Vigil using the @vigil recheck command directly in a PR comment.

← PreviousGetting StartedNext →Configuration