The Data Contract

Executive summary

Downstream systems don't want a smart model. They want one predictable shape — every time, from every agent, in every line.

Policy administration, rating engines, the data warehouse, and analytics all share a hard requirement: a record must look the same regardless of which agent or line of business produced it. A date is YYYY-MM-DD. Money is a plain number. A yes/no field is exactly "Yes" or "No". Absent data is null, not an empty string or a guess. When those guarantees hold, records drop straight into a pipeline. When they don't, someone writes per-line adapters — and maintains them forever. InsightXtract makes the guarantees a first-class, admin-editable artifact rather than something rediscovered in each integration.

1. The normalization tax

Ask an underwriter for a policy effective date and you might get 3/1/25, March 1, 2025, or 01-Mar-2025 — all correct, all different. Ask for a limit and you might get $50,000,000, 50M, or 50,000,000.00. A quota-share percentage might arrive as 46.5% or 0.465. A "claims-made?" flag might be Y, true, Yes, or a checkbox glyph. Each of these is a faithful reading of some source document. None of them is a contract.

The cost of that variability is real and it lands squarely on the integration team:

• Per-line adapters. Every new agent or line of business risks a new set of formats, so someone writes a mapping layer to normalize each one before it reaches the warehouse or the policy system.
• Silent breakage. A rating engine that expects a decimal ratio and receives "46.5%" either errors or, worse, silently mis-rates. A string where a number was expected corrupts an aggregate.
• Untestable output. When shape depends on which model ran, which prompt fired, and which document it read, you cannot write a stable schema test against it.

The fix is not "prompt the model harder." The fix is to define the target shape once, apply it to every agent, and enforce it deterministically so conformance never depends on model behavior.

2. The canonical rules

The output-format contract is type-driven: every field has a type, and each type has exactly one canonical representation. The rules are deliberately boring — boring is what makes a downstream schema stable.

Type	Canonical rule	Before → After
Date	ISO calendar date, YYYY-MM-DD	March 1, 2025 → 2025-03-01
Money	Full number, no currency symbol, no thousands separators	$50,000,000 → 50000000
Boolean	Exactly the strings "Yes" or "No"	Y / true → Yes
Percentage	Decimal ratio, not a percent-styled string	46.5% → 0.465
Not found	Explicit null — never "", "N/A", or a guess	(absent) → null

These five rules cover the overwhelming majority of the ambiguity that breaks downstream loads. A monetary value is always a number you can sum. A percentage is always a ratio you can multiply. A boolean is always a value you can filter on. And a missing value is always the same thing — a true absence, not an invented placeholder that quietly pollutes your data.

3. Two enforcement layers: instruct, then enforce

A model can be asked to emit canonical values, and it will comply most of the time — but "most of the time" is not a contract. InsightXtract therefore uses two layers so conformance is guaranteed regardless of which model runs.

Layer 1 — instruct the model

The canonical rules are passed to the LLM as explicit formatting instructions during extraction. This steers the model to emit values already close to canonical form: it renders the date as YYYY-MM-DD, drops the currency symbol, and returns null for what it cannot find. Because the model produces near-canonical output, the second layer has less to correct and disagreements are rarer.

Layer 2 — enforce deterministically

After extraction, a deterministic post-processing step applies the same contract in code. It coerces $50,000,000 to 50000000, 46.5% to 0.465, and Y to Yes — whether or not the model already did. This layer does not depend on model behavior, prompt wording, or provider. It is the guarantee.

4. Standardize, format, derive, validate

The enforcement layer is not a single function — it is a declarative post-processing SDK. Rules are configured, not coded, and come in four kinds that run in a defined order and produce a standardization report for every run.

Rule kind	What it does	Example
Standardize	Map raw values to canonical terms via glossaries and lookup tables	Calif. → CA; Fully funded → adequate
Format	Type-driven canonicalization from section 2	$50,000,000 → 50000000
Derive	Compute fields from other fields	attachment + limit → total exhaustion point
Validate	Enforce per-field and cross-field invariants	excess limit ≤ underlying; effective < expiration

Standardize is where free text becomes controlled vocabulary: US state codes, coverage types, claim statuses, and domain-specific mappings like "Fully funded" → "adequate" all resolve through glossaries and lookup tables. Format applies the canonical type rules. Derive fills in computed fields so downstream systems don't recompute them inconsistently. Validate catches the records that are individually well-formed but jointly impossible — the kind of error that otherwise surfaces as a rating exception weeks later.

Every run emits a standardization report recording what each rule changed, which lookups resolved, and which validations passed or failed — so the transformation from raw reading to contracted record is itself inspectable, not a black box.

5. Confidence as a first-class field

A downstream system needs more than a value; it needs to know how much to trust it. A raw numeric score is hard to route on consistently across agents. So beyond the score, every field carries a confidence category drawn from a fixed taxonomy:

• exact — the value is stated directly in the source.
• implied — strongly supported by the source but not verbatim.
• inferred — reasoned from related content.
• uncertain — low support; a human should look.

Categories are assigned using configurable confidence bands, with downgrade-only reconciliation: when signals disagree, the field takes the more cautious category, never a more optimistic one. Because the category travels with the field into the output record, downstream routing is consistent regardless of which agent produced it — exact and implied fields can auto-accept into the pipeline, while inferred and uncertain fields route to review. The routing logic lives once, in the consumer, and works the same for every line.

6. Configure once, apply everywhere

The contract is not hardcoded. It is stored in the database, edited by administrators in the console UI, and exposed via an API — a governed artifact rather than a constant buried in code. It is defined globally so a single change propagates to every agent, and it is layered and overridable:

• Global default. The canonical rules apply to every agent's output by default.
• Per-agent override. An agent that needs different behavior — a line that reports rates as basis points, say — overrides just the rule it needs.
• Additive rules. An agent can add extra standardize/derive/validate rules on top of the global contract without forking it.

The result is the property integration teams actually want: change the contract in one place, and every downstream record changes with it — no redeploy, no per-line glue code, no drift between agents. A canonical output record looks the same whether it came from a D&O agent or an excess-casualty agent:

InsightXtract — agentic document extraction for specialty P&C. The output-format contract, post-processing SDK, and confidence taxonomy described here reflect the platform configuration at time of writing.