Self-Checking Extraction: Cross-Field Validation and the Re-Extract Loop

A confident wrong answer is worse than an obvious blank. If an extraction tool reports total_incurred: $310,000 at 0.9 confidence, and the per-claim rows actually sum to $312,500, you now have a clean-looking number that’s wrong — and nothing flags it until a claim, an audit, or a reinsurer does. The fix isn’t a better guess. It’s giving the system a way to check itself against what it already knows must be true.

InsightXtract’s validate step does exactly that, and it’s a first-class part of the extraction loop — not a downstream QA report. Two kinds of checks run on every result, and a genuine second attempt follows any failure.

Two kinds of checks

Per-field checks — is this value even plausible?

Each field is validated against its own spec: is a required field present, is a number a number, a date a date, a value within range, a code found in its reference list?

Rule	Checks	Example
`required`	The field must be present	`policy_number` can’t be blank
`range`	Numeric bounds	`premium` between 0 and 10,000,000
`lookup_match`	Value exists in a reference table	`insured_state` is a real US state code
`glossary_match`	Value maps to a known business term	`coverage_type` is a recognized coverage

Cross-field invariants — do the values agree with each other?

This is where domain judgment lives. An invariant is a rule that must hold across fields or table rows — the kind of arithmetic and logic an experienced reviewer checks by reflex. InsightXtract encodes them declaratively, so the agent checks them on every run.

Loss run must foot Σ(claims.incurred) == summary.total_incurred

The per-claim incurred figures must sum to the printed total. Caught: rows summed to $312,500 against a printed $310,000. The agent re-reads the table, finds a row it merged, and corrects it — or, if the document itself is inconsistent, keeps the printed values and flags does_not_reconcile rather than inventing a fix. error

Policy dates must order effective_date < expiration_date

A term can’t end before it starts. Caught: a mis-read 07/01/2026 – 01/07/2026 from an ambiguous scan. The agent re-extracts both dates with vision and resolves the correct order. error

Roll-ups are computed, not trusted total_payroll == Σ(exposure.payroll)

High-stakes totals — TIV, payroll, exposure — are derived from the rows and checked against the document’s own summary. Caught: a stated payroll that didn’t match the state-by-state schedule; surfaced as a warning for the underwriter with both numbers shown.

flowchart LR E[extract] --> V{validate
per-field + invariants} V -->|all pass| R[reflect & ship] V -->|failure
retries < 2| RE[re_extract
ONLY failed fields
sharpened prompt · temp +0.1] RE --> V V -->|failure
retries used| F[keep values +
flag for review]

The re-extract loop: a targeted second attempt

Detecting an error is only half the value. When a check fails and the agent hasn’t used its retry budget, it loops back and re-extracts only the fields that failed — not the whole document — with a prompt sharpened to the specific problem and a slightly higher temperature to escape the earlier reading. Then it re-validates. The retry budget is bounded (two passes), so the loop always terminates.

Two outcomes, both safe:

The re-extract fixes it. The table now foots, the dates order correctly — and the run proceeds with the corrected values and a note in the trace.
It can’t be fixed (the document is genuinely inconsistent). The agent keeps what the document actually says, attaches the failed check, and routes it to review. It never rewrites a figure to force a rule to pass — the same “flag, don’t fix” discipline we apply to cross-document conflicts.

Checking a value against the label beside it

A form prints limits, retentions and premiums on adjacent rows of the same block. Flatten that to text and the rows lose their column — which is how a line reading Premium $0.00 ended up in a field asking for an employment practices retention, cited to the right page with the right coordinates, and wrong.

The check needs no extra model call, because the pairing already exists: form OCR returns each value together with the label printed beside it. If the schema wants a retention and the form calls that number a premium, the two disagree about what kind of figure it is, and the value is flagged and its confidence dropped.

The first version of this was too aggressive, and it is worth saying how. It asked whether the field name and the form label shared any word. But forms label the coverage — “Employment Practices Liability” — while a schema names the metric — epl_per_claim_limit. Those share nothing, so correct pairings looked like mismatches: run in clearing mode it removed ten right answers to catch one wrong one. Narrowed to a genuine contradiction — the label names a different kind of figure than the field — it caught the real error and left the rest alone.

So it ships off, and the recommended setting is warn: annotate the field and lower its confidence rather than empty it. A check that silently deletes correct data is worse than the error it prevents.

Severity decides the destination

Every rule carries a severity. An error that survives re-extraction holds the document back for review; a warning lets it proceed but travels with the result so an underwriter sees it. This is what makes the straight-through-vs-review decision objective instead of a vibe.

InsightXtract results list — extracted documents with confidence scores and review status, so flagged items surface for an underwriter while clean ones clear automatically — Validation outcomes drive the queue: clean, high-confidence results clear; anything that failed a check surfaces for review.

Where the rules come from

Invariants and field rules aren’t hard-coded in the engine — they’re part of each document class’s configuration, declared once and applied on every run:

// in a document class / master_config — declarative validation
validate:
  - { field: fields.policy_number, rule: required, severity: error }
  - { field: fields.premium, rule: range, min: 0, max: 10000000, severity: warning }
invariants:
  - { name: loss_run_foots, rule: "sum(claims.incurred) == summary.total_incurred", tolerance: 1 }
  - { name: dates_order, rule: "effective_date < expiration_date" }

Because they’re configuration, a business analyst can add or tune a check in the Rules tab — no code, no deploy — and it takes effect on the next extraction. Domain teams keep encoding the checks they’d otherwise do by hand, and the agent keeps applying them at scale.

Why it matters

Errors are caught where they’re cheapest — before a human, before a downstream system. A foot-check that fails at extraction never becomes a mispriced quote.
Confidence you can act on. Because values are validated against rules and evidence, a high score genuinely means “safe to automate,” which is the whole basis of straight-through processing.
Reviewers see problems, not everything. The system points a human at the fields that failed a check — not at a wall of green fields to re-read.
The document stays true. Keeping inconsistent figures and flagging them — rather than silently “fixing” — is what keeps the record defensible.

Self-checking extraction: cross-field validation and the re-extract loop