Agentic AI System for Producer Settlement Reconciliation

For the Controller, reconciliation had become one of the most time-consuming parts of the monthly close.

One or two employees spent a significant share of the monthly close checking which deliveries belonged to each contract, which quantities had already been settled, and whether the final payout followed the applicable terms. A single contract could cover several deliveries. Settlements could span multiple periods. Adjustments could change the billable quantity or final amount.

The difficulty was distinguishing genuine financial discrepancies from differences caused by valid contract terms, partial settlements, or timing across accounting periods.

As the workload grew, the Controller needed a faster way to reach verified totals within closing deadlines. The request was straightforward: “identify the discrepancies automatically.”

The architecture separated probabilistic document interpretation from deterministic financial execution. LLM components handled document understanding, ambiguous matching, and explanation generation, while application services executed settlement rules, monetary calculations, validation, and persistence.

• Unified reconciliation pipeline. Devox brought weight tickets, contract PDFs, and settlement files into a single processing flow within the client’s existing accounting environment. Every file received a unique ID, processing state, content hash, and version reference. Raw documents remained available for reprocessing and audit lineage. Python services formed the application layer around the workflow, while PostgreSQL persisted document state, reconciliation runs, and the relationships between deliveries and settlements. The existing accounting environment remained the system of record, while the new application operated as a reconciliation layer around it.
• Field-level extraction with provenance. Structured files passed through tabular parsers, while scanned documents used OCR and layout analysis with LLM-assisted extraction. OpenAI handled the language-dependent extraction step for variable document structures. The model output entered the reconciliation workflow only after application-level normalization and validation against the expected document schema. Each extracted value retained its page reference, source location, raw text, and extraction method. Numeric validation checked units, allowed ranges, and weight consistency before records entered the reconciliation workflow. A low-confidence reading therefore stopped at the document-processing boundary instead of propagating into a settlement calculation. The system modeled contracts, contract versions, delivery tickets, settlement lines, allocations, reconciliation runs, and review decisions as separate PostgreSQL entities. A dedicated Allocation record stored the exact quantity from each delivery already applied to a settlement.
• Four-pass matching instead of a single fuzzy search. The engine started with exact ticket and contract references. It then narrowed candidates by producer and date window. Attribute scoring evaluated supporting evidence such as volume or location. Combinatorial matching handled cases where several delivery tickets collectively matched one settlement line. AI assisted with ambiguous descriptions, while matches were automatically accepted only when they met validated confidence thresholds.
• Deterministic settlement calculations. After matching, the calculation engine reconstructed the expected payout from the active contract version: Billable Quantity × Applicable Price + Additions − Deductions. Decimal arithmetic with explicit precision and rounding rules kept monetary calculations reproducible. Explicit rules governed billable quantity, unit conversion, contract-effective dates, adjustment logic, and rounding points. LLM-extracted contract terms entered execution only after mapping into validated structured rules.
• Constrained agent orchestration. OpenAI provided the reasoning layer, while Python application services exposed a narrow set of domain operations for ticket candidate retrieval, contract-term lookup, settlement calculation, and source-evidence retrieval. The model could choose which operation to invoke and use its result to prepare an explanation.
• Evidence-backed exception handling. Each flagged case showed the linked source evidence and the calculation applied. The reviewer could see the expected value alongside the settlement value, understand the discrepancy class, and decide whether to confirm the match, change an allocation, or escalate the case. Allocation changes triggered recalculation of the affected settlement records.
• Revision-aware reconciliation. Updated tickets or settlement files triggered reprocessing only for the affected record chain. The AWS deployment ran this as stateful application processing rather than regenerating the full reconciliation from scratch. PostgreSQL preserved the dependency chain needed to identify which allocations and settlements had become stale. Each reconciliation snapshot preserved source document versions, accepted mappings, rule-engine version, and the AI component version involved in the run.

The clearest outcome was a shift from record-by-record reconciliation to an exception-based review process. Routine matches could move through the workflow automatically, while the Controller’s team focused on cases with genuine financial or documentation-related uncertainty.

Technical Results

• Probabilistic AI was isolated from financial execution. OpenAI could interpret documents and resolve ambiguous cases, while deterministic application code retained authority over payout calculations and validation.
• More reconciliation logic became machine-executable. Delivery matching, contract application, settlement validation, and discrepancy classification moved into a repeatable workflow instead of depending on manual reconstruction during close.
• Partial and multi-period settlements became traceable. Allocation-level records preserved how much of each delivery applied to a settlement, helping prevent repeated use of the same ticket across reconciliation runs.
• Financial calculations became reproducible. Expected settlement amounts were derived through deterministic rules with explicit precision controls, giving reviewers a stable basis for comparing expected and recorded values.
• Document evidence stayed attached to every result. Extracted fields retained provenance back to the source document, which made flagged variances easier to verify during accounting review.
• Reconciliation became revision-aware. Updated tickets or settlements could trigger targeted reprocessing while preserving historical run state, rule versions, and prior review decisions.
• Review quality became measurable at several levels. The evaluation model measures OCR accuracy, match precision, false-negative rates, calculation accuracy, and rerun stability, rather than reducing system quality to a single AI accuracy score.

Business Results

• ~70% less manual reconciliation effort. Routine matches moved through the workflow automatically, allowing the finance team to spend most of its review time on genuine financial or documentation-related exceptions.
• ~80% of routine settlement cases processed without manual reconstruction. Exact-reference matching, candidate filtering, attribute scoring, and grouped matching handled standard cases before human review.
• ~50% faster reconciliation during monthly close. The Controller reached verified settlement totals earlier because the team reviewed exceptions instead of manually rebuilding every delivery-to-contract-to-settlement relationship.
• Faster exception investigation. Reviewers received the expected value, recorded settlement value, discrepancy class, source evidence, and applied calculation in the same workflow.
• Higher capacity during seasonal peaks without proportional team growth. The workflow could absorb larger document volumes while keeping manual effort concentrated on exceptions.
• Earlier visibility into payment discrepancies. Missing deliveries, duplicate tickets, rate differences, and adjustment issues could surface before payment approval, giving Finance more time to resolve them within the close cycle.
• Clearer visibility into open producer liabilities. Finance could separate reconciled settlements from pending documentation and active discrepancies instead of reconstructing that status manually.
• Automation without replacing the accounting platform. The client added a reconciliation layer around the existing system of record, avoiding the cost and risk of replacing the entire commodity accounting system.