Why a field description won’t stop a model copying the wrong number

Schema descriptions are the main lever most extraction platforms give you. Write a better field note, get a better answer. It works — up to a point, and the point is sharper than we expected. Over a few weeks of tuning two production agents against real submissions, the same pattern appeared four times: a description resolved what a field meant, and did nothing at all to stop the model reaching for the nearest plausible number.

Four times it failed the same way

A management liability application states, on one line, Total number of Applicant’s voting shareholders: 84. Elsewhere, a schedule lists the board — five directors, by name. The schema asked for both figures, and the description for each said explicitly what the other one was:

“Distinct from number_of_shareholders: a board of 5 in a company with 84 shareholders is 5, and the two fields returning the same figure is the commonest error here.”

It returned 84 for both. The description named the exact failure, and the failure happened anyway.

Same story elsewhere. A form requests one shared aggregate across its coverage sections and states no per-claim limit at all; we said so, and the per-claim field came back with the aggregate in it. A broker’s email named the brokerage but no individual; we said “leave this empty where no individual is named,” and it filled the field with the insured’s own CFO. A retention line stating three figures at once — $20K - D&O, $50K - EPL, and $0 - FID — had two of the three read correctly and the third filled from a different document’s premium line.

Why prose loses

Nothing here is a comprehension failure. At the moment of answering, the model has a labelled number in front of it and a field that plausibly wants a number. A sentence written into a schema competes against a figure sitting right there on the page, and the figure is more immediate. Worse, the wrong answer arrives with a citation — page, coordinates, the verbatim quote — because the number really was on the page. It simply belonged to something else.

That is what makes this class of error expensive. It does not look like a gap. A blank field asks to be checked; a confidently cited wrong number does not.

What worked instead

In every one of the four cases the fix was the same shape: stop asking, start computing.

• Count the rows. number_of_directors is now derived from the directors schedule. Five rows, five directors. The shareholder figure cannot get in.
• Parse the line, don’t re-read it. The three retentions are extracted from the one line that states them, with the figure matched to its coverage by the punctuation that joins them — not by proximity, which picks the wrong one when a form writes D&O Retention: $25,000 / EPL Retention: $10,000.
• Refuse a value that equals its neighbour. Where a submission requests a shared aggregate, a per-claim limit identical to the aggregate was copied from it, and is cleared.
• Use the evidence already in the record. When the broker and the insured’s contact came back as the same person, the contact’s email domain settled which side they were on — deterministically, with no second model call.

Each of these is a rule in the post-processing config, not code. Each records why it fired, so a reviewer sees “counted from the claims table” rather than a number that changed for no visible reason.

The rule we follow now

A description resolves what a field means. It does not prevent a model copying a nearby number. Anything computable from data already extracted should be computed, not described.

Descriptions still matter enormously — for the other half of the problem. When we audited one agent, 32 of its 118 fields had no description at all, and three of the four disagreements we chased between different extraction engines traced back to exactly that: two readers answering two different reasonable questions. org_type meant “which D&O market segment” to one and “how is the company legally constituted” to another. Both are sensible readings of a bare field name. Only a description settles it — and the fix was to describe one field properly and add the second field the confusion revealed was missing.

How to tell which you need

Symptom	Reach for
Two readers give two different but defensible answers	A description — the field is ambiguous, not misread
A field holds a figure that belongs to a neighbouring field	A rule — describing the difference will not stop the copy
A header figure disagrees with the table beneath it	A derivation — count or sum the rows
A value is right but written three ways across submissions	A glossary or format rule
A field is blank that a document plainly answers	A description, or a missing table declaration

The uncomfortable corollary

Rules can be wrong too, and ours were — more than once. An early version of a label check, meant to catch a premium read as a retention, cleared ten correct values to catch one wrong one, because it required a field name and a form label to share a word, and forms label the coverage while schemas name the metric. A rule intended to answer “no coverage requested” from the absence of a limit asserted eight false Nos on a form that simply stated no limits anywhere.

So every rule that can empty or overwrite a field ships off, and every one of them logs what it did and why. A rule you cannot see is worse than a description you can. The point is not that computation is safer than prose — it is that computation is checkable, and prose about a number is not.