The architecture behind hard P&C documents: how InsightXtract reads what breaks everything else

P&C submission documents are, quietly, one of the hardest data-extraction problems in the enterprise. Not because any single page is impossible — because the population is adversarial. A loss run whose subtotals don’t foot. A statement of values that runs the same table across nine scanned pages. A bordereau half in English and half in Spanish. An exposure workbook where every broker invents a new column order. A 40-MB PDF with a chart that holds the only copy of a number you need. Ordinary IDP is built for the clean 5% and silently guesses on the other 95%.

InsightXtract is built the other way around — for the 95%. It is not a model with a prompt; it is a modular, governance-first, agentic architecture where every hard case has a named handling path, every value carries its evidence, and every behavior is something you configure rather than hope for. This piece walks the whole architecture: the reasoning loop at the core, the agent-of-agents above it, the domain skills that make it an expert, the handling paths for the ugliest documents, the governance spine that guarantees consistency, the token economics that make it affordable, and the auditability that makes it trustworthy. With examples throughout.

Why P&C documents break ordinary tools

Before the architecture, the enemy. These are the failure modes we designed against — each one has a dedicated path later in this article.

The architecture at a glance

Before the details, the map. InsightXtract is two planes over a model-agnostic core: a data plane that moves a submission from raw files to a governed record, and a control plane that decides how — skills, vocabulary, contracts, permissions and the audit trail. Nothing in the data plane is hard-coded; every step is steered by configuration in the control plane, which is exactly why a new line of business is a configuration change, not a new pipeline.

Two planes over a model-agnostic core. Swap the model, add a line of business, change the vocabulary — the data plane doesn’t change; the configuration does.

The core: a reasoning loop, not a single pass

At the heart of every extraction is an agent that reasons in a loop — a state machine with five moves and a self-correction cycle. It doesn’t finish until it has checked its own work.

Worked example — a loss run that doesn’t foot. The agent perceives a 5-page claims table with a totals row. It plans to read the table with layout-aware extraction and the header fields as text. It extracts 214 claim rows and a “Total Incurred” figure. In Validate, an invariant fires: sum(paid + reserve) == total_incurred — and it’s off by one claim. Instead of shipping a wrong number with a green checkmark, the agent re-extracts the rows on the page where the delta lives, finds a row split across a page break, corrects it, and only then compiles the record — now footing, marked high-confidence. A single-pass tool returns the wrong total and never knows.

An agent of agents: classify → typed extract → consolidate

A real submission is not one document. So above the reasoning loop sits an orchestration graph that treats the packet as a whole.

• Classify & route. Each file is typed — submission email, ACORD 125/127/140, SOV, loss run, financials, 10-K, supplemental — using filename and first-page evidence, and routed to the right handler.
• Per-document, per-type extraction. Each document is read by a strategy matched to it: a spreadsheet analyzer for Excel, native document reading for born-digital PDFs, vision for scans and charts, text for email. Each produces a typed, cited partial record.
• Consolidation. The partials are merged into one unified record by an explicit priority policy (the ACORD wins for the FEIN, the workbook wins for TIV, the 10-K wins for financials), with conflicts surfaced rather than silently overwritten — and every consolidated field remembers which document it came from.

Example. A Public D&O submission arrives as an email + application PDF + financials workbook + ACORD. The orchestrator classifies all four, extracts each with its own strategy, then consolidates into a single record: insured, ticker, CIK, a 9-row board-and-officers table (with each director’s other public boards), three years of fiscal figures, and a claims history — each value tagged with the source document and page. One packet in, one clean, cited, decision-ready record out.

A domain-aligned entity graph: accuracy from structure, not just text

Here is a lever most extraction tools never pull. InsightXtract doesn’t extract fields into a flat bag of strings — it extracts into a domain entity model: an insured, its locations, coverages and limits, its claims, its directors and officers, its broker. Because the target is a typed, related graph rather than free text, the agent gets a second source of accuracy for free — the shape of the domain itself.

• Cross-document entity resolution. The “Acme Corp” in the email, the “Acme Corporation, Inc.” on the ACORD, and the FEIN on the 10-K are resolved to one insured node — so conflicting values sit side by side on the same entity instead of scattering into three half-records.
• Relationship validation. A limit must attach to a coverage; a claim must belong to a policy period; a director’s other-boards must resolve to real companies. The graph rejects a value that can’t be placed — a structural check text-only extraction can’t make.
• Ontology-constrained decoding. Because the model knows it is filling a coverage or a loss, not “some number,” it reads the right cell for the right reason — the domain model prunes the wrong answers before they’re written.

Prioritization — when documents disagree, the right source wins. A submission is full of the same fact told several ways. Consolidation into the entity graph is governed by an explicit source-authority and confidence policy: the ACORD wins for the FEIN, the audited financials win for revenue, the exposure workbook wins for TIV, the broker email loses to all of them. Where authority ties, the higher-confidence, better-cited value wins — and the disagreement is recorded on the entity, not silently overwritten, so a reviewer sees exactly why the record says what it says.

Dynamic domain skills: the playbooks that make it an expert

A general model knows what a table is. It does not know that on an ACORD 140 the Coverage B limit is per-building, that a workers-comp class code maps to a payroll basis, or how a Public vs. Private vs. Non-Profit vs. Financial-Institution D&O submission differs. That expertise lives in domain skills — modular, progressive-disclosure playbooks the agent loads only when relevant.

• Progressive disclosure. The agent doesn’t carry every playbook in context. It loads the skill for the document class in front of it — keeping prompts small, focused and cheap.
• Configurable, not hard-coded. Skills are authored as document-class specs (fields, tables, per-field hints, bound glossaries and lookups, validation rules) — edited in the console, versioned, and reused across agents.
• Composable. A new line of business is a new set of skills over the same engine — not a new codebase.

The hard-document playbook

Now the ugly cases — and the concrete path for each.

Over-running & complex tables

The agent reconstructs a table as one logical object across page breaks: it carries the header context forward, re-anchors columns on each page, stitches rows split by a page boundary, and understands merged cells, sub-headers and per-row currencies. Totals are validated against the rows (footing), so a schedule that runs 40 pages produces one clean table with a total that reconciles — not nine disconnected grids.

Charts, graphs & images

When Perceive detects that the data lives in a figure — a loss-development triangle, a bar of premium by year, an org chart — Plan routes that region to vision. The model reads the graphic directly and returns structured values, cited to the page. The number that only ever existed as a chart bar becomes a field with provenance.

Badly scanned documents

Skew, speckle, stamps and fax headers are expected, not exceptional. The agent uses image-based reading where OCR text is unreliable, and — crucially — when a field can’t be read cleanly, it says so: the value is returned low-confidence with its snippet and routed to a human, rather than shipped as fact. Silence about uncertainty is the failure mode we refuse.

Mixed & partial-language handling

Documents aren’t assumed to be monolingual. The agent handles pages (or regions) in different languages, translating for understanding while preserving the original text as evidence, so a bilingual bordereau or a foreign-language supplement is extracted in full — and the underwriter can still see the source words behind every value.

Dynamic Excel

Spreadsheets get their own strategy because they deserve one: rather than assume a layout, the agent inspects each workbook and writes a parser for it — the single most important trick for beating broker-variable data. It’s important enough that it gets its own deep-dive in the next section.

Very large, scanned files — hundreds of pages, hundreds of megabytes

A single submission can arrive as a 500-page, 200-MB scanned PDF — a stack of ACORDs, a 60-page schedule of values, years of loss runs and a full policy wording, faxed and re-scanned until there is no text layer at all. That breaks ordinary tools three ways at once: it won’t fit any context window; reading every page with vision is slow and expensive; and 90% of it is boilerplate you don’t need. Size is treated as a routing problem, not a wall:

• Page-level triage first. The document is classified page by page — this page is an ACORD 125, these forty are the SOV, this block is the policy wording — so the engine spends effort where the answers are and skips the filler.
• Retrieve, don’t re-read. Content is chunked and indexed; the agent retrieves only the passages relevant to each field and reads those at full fidelity, instead of pushing 500 pages through a model.
• Vision only where it’s earned. Scanned regions get image-based reading; clean regions get cheap text — so you never pay vision prices for a page that didn’t need it.
• Parallel & streaming. Pages and documents are processed concurrently and results stream back, so a huge packet finishes in wall-clock minutes, not a serial crawl.

The outcome: complete coverage of a document nobody could paste into a chat window — at a fraction of the cost of reading all of it, with every extracted value still cited to its exact page.

Dynamic code generation: the agent writes a parser for every workbook

This is where the hardest documents are actually beaten, so it earns its own section. Broker spreadsheets and multi-page schedules have no stable layout — the header is on row 3 or row 11; loss years run across columns on one workbook and down rows on the next; a “values in USD 000s” note tucked in a merged banner silently rescales an entire sheet. You cannot write one parser for that population. So the agent doesn’t: it inspects each document and writes the parser for that document, runs it in a sandbox, checks the output against the target schema, and repairs the code if it fails — a code-generation reasoning loop.

The code-generation loop — the agent writes, runs, checks and fixes a parser per document.

What the agent actually produces is not a prompt over pasted cells — it’s real, inspectable extraction code, generated against the structure it detected:

# Sensed: header on row 11 · merged title band A1:H9
# "Values in USD 000s" -> scale x1000 · TIV = Building + Contents + BI
def extract(wb):
    df = pd.read_excel(wb, sheet_name="Schedule of Values", header=10)
    df = normalize_headers(df)                 # fuzzy map -> canonical fields
    df["tiv"] = (df["building"] + df["contents"] + df["bi"]) * 1000
    assert df["tiv"].sum() == scale(read_control_total(wb))   # must foot
    return df[["location_id", "address", "state", "tiv", "construction", "year_built"]]

A parser the agent wrote for one broker’s SOV — detected layout, generated code, sandbox result, and the footing check that gates it.

Because it is code, it is reproducible and reviewable: the same workbook yields the same output every time, the logic can be audited, and the footing assertion is a hard gate — a sheet that doesn’t reconcile never passes silently.

Complex patterns the code-gen layer handles

Before and after — one messy sheet, one clean record:

Merged title, “000s” scaling and abbreviated headers → a tidy, typed, footing schedule — every value cited back to its cell.

The same idea beats complex PDFs

Code generation isn’t Excel-only. When a table runs 40 pages across a scanned schedule, the agent reconstructs the grid and generates the stitch-and-reconcile logic — carrying the header forward, re-anchoring columns per page, joining rows split by a page break, then footing the total against the rows. And derived figures never rely on the model doing arithmetic in its head: totals, loss ratios and rollups are computed by generated, checkable code over the extracted rows, so the header number equals the sum of the detail — exactly, every time.

Governance-first, by construction

Everything above would be fragile if the meaning of the output drifted. It doesn’t, because the schema and the vocabulary are governed, not emergent.

• Document classes & specs define exactly what a class of document yields — fields, tables, types — so every ACORD 125 in your book returns the same shape.
• Glossaries & reference lookups standardize values: California → CA, a description → the right NAICS/SIC/ISO class code, a coverage synonym → the canonical term — with semantic lookup for the fuzzy cases.
• Output contracts pin formatting globally: dates, money, percentages, booleans and nulls come out one way, so fiscal_years is always a typed, ordered series — never a pile of strings.
• RBAC & versioning mean agents and outputs are permissioned and pinned; a published version keeps returning yesterday’s shape until you promote a new one.

Adaptive token models: accuracy you can afford

Reading everything with the biggest model at full fidelity is accurate and unaffordable. InsightXtract makes cost a first-class design axis.

• Per-field extraction mode. Each field is read the cheapest way that works — text for clean born-digital passages, textract_text for OCR, vision only where pixels are the source of truth. You don’t pay vision prices to read an email.
• Prompt caching & single-path routing. Stable instructions and skills are cached across a run; the orchestrator avoids redundant passes so a document isn’t read three times.
• Model tiering. Cheap, fast models handle classification and clean fields; the strongest model is reserved for the genuinely hard extraction and the consolidation reasoning.
• Retrieval over brute force. On large files, reading only the relevant chunks turns a whole-document bill into a fraction of it.

The result is a curve you can dial: the same engine can run a high-accuracy, cost-is-no-object pass for a bound account and a lean pass for triage — without changing the pipeline.

Auditability, guardrails & citations

None of this matters to a regulated underwriter unless they can trust and defend it. So trust is built in, not bolted on.

• Citations on every value. Each field carries provenance — source document, page, bounding box and the text snippet it came from. A reviewer clicks a number and lands on the exact spot in the exact page.
• A confidence taxonomy. Confidence isn’t a mystery float; it’s a defined scale that drives routing — high-confidence values flow straight through, low-confidence values stop at a human-in-the-loop gate.
• The glass-box trace. Every run records what the agent saw, how it planned, what it validated, what it re-extracted and why — a replayable audit trail, not a black box.
• Guardrails & evaluation. Rules constrain outputs; gold datasets and an eval harness measure accuracy per field and per document class, so a change is a measured improvement, not a hope.

Measured on a synthetic-but-realistic gold packet (email + application + financials + loss run). Residuals route to a human, not to production.

And every one of those values is clickable back to its source. This is what a cited field looks like — the number, the exact document and cell it came from, its confidence, and the snippet of text behind it:

A reviewer clicks the figure and lands on cell C14 of the workbook — provenance, not a promise.

How it fits your stack

Architecture that can’t be operated isn’t architecture. InsightXtract is built to drop into a regulated insurer’s environment, not to replace it.

• API-first & agent-invokable. Every capability is a permissioned API with scoped keys and usage metering — call it from your PAS, your underwriting workstation, or another agent.
• Runs where your data lives. Managed cloud, or deployed into your own VPC / on-prem as a container — the same engine, your data residency.
• Model-agnostic. The reasoning loop isn’t welded to one LLM; models are tiered and swappable, so you’re never locked to a single vendor or price curve.
• Secure code execution. The parsers the agent generates run in an isolated sandbox — no network, no filesystem, hard resource limits — so dynamic code is contained by construction.
• RBAC, versioning & audit by default. Roles gate who configures agents; published versions pin the output shape; every run is traced and every value cited — the trail your model-risk and audit teams already ask for.

Why modular + governance-first guarantees a better outcome

Pull it together and the guarantee is structural, not aspirational:

Ordinary IDP optimizes one number on clean documents. InsightXtract optimizes the outcome on the documents you actually receive — and shows its work. That is the difference between a demo and a system you can put in front of an auditor.