LedgerLedger

Invoice processing is usually described as a reading problem. In practice it is a trust problem. A system that is right 95% of the time and cannot tell you which 5% it got wrong is unusable for accounting, because every record has to be checked anyway — which is the work you were trying to remove.

The interesting question is not “how accurate is extraction?” but “can the system reliably identify its own failures?” A pipeline that is right 90% of the time and knows exactly which cases it is unsure about removes far more work than one that is right 97% of the time and is uniformly confident.

Inconsistent inputs

Native PDFs, scans, photographs of printouts, and multi-invoice attachments in a single file.

Non-negotiable fields

Bank details may never be auto-populated from a document, regardless of extraction confidence.

Idempotency

The same document frequently arrives twice. Posting twice is worse than not posting at all.

Explainability

Every posted figure must be traceable to a page region in the source file months later.

We treated the language model as one component in an ordinary data pipeline rather than the pipeline itself. Documents are ingested, hashed and classified first. Extraction runs field by field against an explicit schema, and the model is required to emit a confidence value and permitted to emit null with a reason.

Everything after extraction is deterministic code: schema parsing, arithmetic checks, supplier reconciliation, policy rules. Deterministic checks are the part of the system that makes it trustworthy, because they fail the same way every time.

What the model cannot resolve becomes a review task scoped to the specific uncertain field — not the whole document. Reviewer corrections are stored and retrieved as precedent the next time a document from that supplier appears.

Ingestion, extraction and validation are separate workers communicating through a queue, so a slow OCR pass cannot block validation of documents already extracted, and any stage can be retried independently.

Ingest worker

Normalises file types, hashes content for idempotency, splits multi-invoice attachments, stores page images.

Context assembly

Loads the target schema version, the finance policy set, and retrieved precedent for the matched supplier.

Extraction worker

Field-level extraction with per-field confidence and page-region citations.

Validation service

Pydantic schema parse, line-sum arithmetic, supplier reconciliation, threshold policy — pure functions, fully unit tested.

Review queue

Field-scoped tasks with the source region highlighted; corrections written back as retrievable precedent.

Ledger writer

Single idempotent write keyed on the document hash, with a stored decision trace.

The reviewer interface is the product. It opens on the one field the system is unsure about, with the relevant region of the page already in view, and resolves in a single keystroke.

We deliberately did not build a chat interface. The reviewer does not want a conversation about an invoice; they want to confirm a number and move to the next one. Designing for the hundredth document of the morning rather than the first changed nearly every interface decision.

Field-level, not document-level extraction

Slower and more expensive per document, but it localises uncertainty — which is the entire value proposition.

Validation in code, not in prompts

Arithmetic and policy rules belong in testable functions. Asking a model to check its own sums is both unreliable and unnecessary.

Corrections as retrieval, not fine-tuning

Precedent retrieval improves behaviour immediately and is inspectable. Fine-tuning would have been slower to iterate and harder to explain.

Write-once with an idempotency key

The document hash is the key. Replays, duplicate emails and worker retries converge on a single ledger entry.

We assembled a fixture set of documents chosen for the ways they break things: rotated scans, tax shown per line rather than per invoice, two invoices in one PDF, credit notes formatted as invoices. Each fixture asserts an expected extraction and an expected routing decision.

The metric we optimised was not raw field accuracy but the rate at which the system silently produced a wrong value that passed validation. Those are the only failures that reach the ledger, and they are the only ones that matter.

Validation logic is covered by conventional unit tests. The extraction stage is held to a regression suite that runs on every change to prompts, schema or retrieval.

This is an experiment, not a delivered engagement, so there are no production figures to report. If it were deployed, these are the measures we would hold it to.

Silent error rate

Wrong values that passed validation, found later by reconciliation. The number that must stay near zero.

Review rate and review cost

Share of documents needing a person, and median seconds spent per reviewed field.

Precedent effect

Whether review rate for a given supplier declines after corrections are stored.

Cost per document

Model and infrastructure spend per processed document, tracked per supplier.