Here is a pattern any underwriting operations leader will recognize. Your team runs excess and umbrella submissions through automated extraction. On the ACORD 131, the named insured keeps coming back as the broker agency — because the producer’s name sits in bold at the top of the form and the model latches onto it. A reviewer fixes it. The next Summit Logistics renewal arrives, and the same field is wrong again. Same mistake, same fix, next week, next quarter, forever.
That is extraction drift in its most expensive form: not a model that gets worse, but a model that never gets better. Corrections are made, value is delivered for that one document, and then the knowledge evaporates. The reviewer’s expertise never makes it back into the system. Multiply one recurring field error across thousands of submissions a year and you are paying for the same correction over and over.
InsightXtract is built to break that cycle. Corrections are not throwaway edits — they are training signal. The platform aggregates them, lets the extraction agent propose concrete configuration changes to address them, and then routes every proposal through a human approval and an automated validation gate before anything reaches production. The result is a genuinely self-improving pipeline where each correction makes the next correction less likely.
The improvement loop, end to end
The core idea is a closed loop. A reviewer corrects a field and, critically, records why. Those reasons accumulate into a feedback set. The agent analyzes the feedback and proposes targeted changes to its own configuration — a sharper field prompt here, a clearer class description there, a new post-processing rule to resolve a recurring conflict. A human reviews the proposal, approves or rejects it, and on approval the platform publishes a new agent version. Before that version can be promoted to production, it must pass validation. Only then does it go live.
flowchart LR
A["Reviewer correction
(field + reason)"] --> B["Aggregate feedback
across submissions"]
B --> C["Agent proposes
config changes"]
C --> D{"Reviewer
approves?"}
D -- "Reject" --> C
D -- "Approve" --> E["Publish new
agent version"]
E --> F["VALIDATE
feedback set, then
golden baselines"]
F -- "Pass" --> G["Promote to
production"]
F -- "Fail" --> C
G -.->|"fewer corrections
next cycle"| A
Notice the dotted line back to the start. That is the whole point: a promoted version means the next batch of Summit Logistics renewals should not reproduce the named-insured error, so the reviewer never has to touch it again. The loop tightens with every cycle.
What a correction actually captures
A correction in InsightXtract is more than an overwritten cell. When a reviewer changes a value, the platform records the field, the old value, the new value, the source document and page, and — the part that makes improvement possible — a short structured reason. “This is the producer, not the insured” is a very different signal from “OCR misread the digit.” The first is a systematic error the agent can fix by changing its instructions; the second is noise. Separating the two is what lets the agent propose changes that actually generalize.
What the agent proposes — three real P&C examples
Aggregated feedback is where drift becomes actionable. When the agent sees the same reason attached to the same field across many documents, it drafts a specific, reviewable change. These are not vague “retrain the model” suggestions — they are edits to the artifacts your team already owns: field prompts, class descriptions, and post-processing rules.
1. A tightened field prompt for named_insured
The named-insured-versus-broker error is a classic prompt problem. The agent proposes replacing the generic instruction with one that encodes the reviewers’ own reasoning about ACORD 131 layout.
named_insured: “Extract the name of the insured party from the form.”
named_insured: “Take the First Named Insured from the ACORD 131 Applicant / Named Insured block. Do not use the Producer or Agency name in the top-right block — that is the broker, not the insured.”
2. A class-description clarification so ACORD 131 vs ACORD 125 route correctly
Some drift is upstream of extraction — it is a routing problem. If ACORD 125 (Commercial Insurance Application) and ACORD 131 (Umbrella/Excess) are being confused, every downstream field inherits the wrong spec. Reviewers flagging “wrong document class” drive the agent to sharpen the class descriptions that the router reads.
3. A post-processing rule for premium conflict resolution
On a real submission the total premium often appears in more than one place — a broker’s cover email might quote a target premium of $148,000 while the ACORD states an annual premium of $152,500. Reviewers consistently keep the ACORD figure. The agent proposes encoding that precedence as a deterministic rule, so the conflict is resolved the same way every time rather than left to the model’s judgment.
The validation gate: feedback set first, then golden baselines
Approval alone does not promote a change. A well-intentioned prompt edit can fix one field and quietly break three others. That is why every proposed version runs through a two-stage validation gate before it can be promoted — and the order of the two stages is deliberate.
flowchart TD
V["New agent version
(approved)"] --> F["Stage 1 - FEEDBACK SET
Re-run the exact documents
reviewers corrected"]
F --> FQ{"Did the corrected
errors stop recurring?"}
FQ -- "No" --> R["Reject / send back
the change did not fix it"]
FQ -- "Yes" --> G["Stage 2 - GOLDEN BASELINES
Re-run the trusted
reference submissions"]
G --> GQ{"Any regression
vs. known-good output?"}
GQ -- "Regression" --> R
GQ -- "Clean" --> P["Promote to production"]
Stage one runs the feedback set first — the exact documents whose corrections motivated the change. This answers the only question that justified the change in the first place: did the fix actually work? Take the Summit Logistics excess-casualty submission that first surfaced the named-insured error. The new version re-runs that document; if named_insured now resolves to the insured rather than the broker agency, stage one passes. If it does not, there is no point checking anything else — the change is rejected and sent back for revision.
Stage two runs the golden baselines — a curated set of trusted reference submissions with known-good, human-verified output. This catches collateral damage: the premium-conflict rule must not, for instance, start discarding legitimate email-only premiums on monoline accounts that have no ACORD. Only a version that both fixes the reported errors and introduces no regressions earns promotion. Running feedback first and baselines second means you never waste a full regression pass on a change that did not even solve its own problem.
Governance: reviewer-approved, versioned, auditable
Self-improvement without governance is just an agent editing itself in the dark. InsightXtract deliberately keeps a human in the loop at the decision point. The agent may propose, but a reviewer disposes — every configuration change is explicitly approved or rejected by a named person, and that decision is recorded alongside the diff and the feedback that motivated it.
Because each promotion produces a discrete, numbered agent version, you always know which version of the configuration produced any given extraction. If an auditor or a downstream dispute asks “why did the system read the premium this way in March?”, you can point to the exact version, the rule that was active, the person who approved it, and the validation run that cleared it. Nothing is silent and nothing is anonymous.
Why the loop is safe to run
The agent never changes itself unilaterally. Proposals are human-approved, versions are immutable and numbered, and no version reaches production without passing the feedback set and the golden baselines. Improvement is continuous; risk is contained.
Why this matters to the business
- Fewer touches per submission, over time. Each recurring error, once fixed at the configuration level, stops arriving in the review queue — so effort per submission trends down instead of staying flat.
- Institutional knowledge is captured, not lost. When a seasoned reviewer knows the First Named Insured lives in a specific ACORD 131 block, that knowledge becomes a durable prompt — it survives them moving teams or leaving the company.
- Change you can defend. Every improvement is approved by a person, tied to the evidence that justified it, and proven safe by a validation run before it ships.
- Compounding returns. Corrections are an asset, not a cost. The more your team reviews, the smarter the agent gets, and the less there is to review.
Related: how it all stays auditable →
Self-improvement is only trustworthy if you can trace every value and every change. The companion post covers auditability by design — who extracted what, when, and why, down to the bounding box on the ACORD.