Invoice field extraction that holds up at posting time
Consider a manufacturer receiving, say, 400 supplier invoices a day, about 60% as scanned PDFs. The AP team is manually keying line items, GSTIN, invoice number, date, and taxable amounts into Tally. The extraction layer we would put in front of that uses a layout-aware vision model — specifically, each page goes to Azure Document Intelligence's `prebuilt-invoice` model, and the raw JSON is stored alongside the PDF in S3.
The hard part is not calling the API. The hard part is deciding which fields are reliable enough to auto-post and which ones route to a review queue. This is easy to get wrong on the first pass.
The model returns a confidence score per field. The obvious first logic is: if every field on the invoice clears 0.85, post it. This falls apart within days. Confidence scores measure the model's certainty that it read the text correctly — not that the text is the right value for that field. Picture a supplier invoice with the HSN code printed in the GSTIN field after a template change. The model reads it with 0.97 confidence. Tally rejects the entry because the GSTIN checksum fails.
We now run three layers of validation before anything posts.
First, structural checks. GSTIN goes through the standard 15-character regex plus the checksum digit algorithm. Invoice dates must fall within the goods-receipt date ± 15 days. PO numbers must exist in the open-PO list, fetched from the ERP via a cached API call. These checks catch the majority of misreads without any human involvement.
Second, cross-field arithmetic. For GST invoices in India, the sum of CGST + SGST (or IGST) must equal the tax amount derived from the taxable value × rate. We compute the expected tax from the line items and compare it to the header tax field. If the difference exceeds ₹1, we flag it. Expect a meaningful share of invoices, somewhere in the range of 5 to 10%, to fail this check — sometimes because the supplier rounded differently, sometimes because the model misread a digit in a line item.
Third, supplier-specific rules. We store a per-supplier template fingerprint keyed on GSTIN. If the model's bounding boxes for key fields shift by more than 15% from the last known template, we route to review. This catches the template-change problem described above.
With these checks in place, we would expect somewhere between two-thirds and three-quarters of scanned invoices to post without a human looking at them. The rest land in a review queue built as a simple React app showing the PDF with bounding-box overlays on the extracted fields. A reviewer sees the model's read, the validation failure reason, and a text box to correct each field. Corrections feed back into a per-supplier accuracy log, which we use to adjust thresholds.
Three fields still go to human review every time, regardless of confidence.
**Line item descriptions.** The model reads them well, but free-text descriptions vary wildly across suppliers — "MS Round Bar 12mm" versus "Mild Steel Round Bar, Diameter 12mm" — and the inventory system needs a specific material code. An LLM-based normalization pass is tempting here. In our experience it gets the mapping right most of the time, but it adds a per-invoice API cost and a few hundred milliseconds of latency, and the misses land directly in inventory. A human picking the material code from a dropdown is usually cheaper than the error rate of automated mapping plus the API cost.
**Bank account numbers on the payment section.** A misread here means money goes to the wrong account. The model reads these with high confidence, but the risk profile is wrong for automation. The reviewer just glances at the last four digits against a stored supplier record.
**Reverse charge flags.** These are usually a checkbox or a Y/N field, but they determine how the entire entry posts. The model gets it right, but the downstream impact of a wrong answer is significant enough that a human confirmation is worth the 3 seconds it takes.
At published rates, the Document Intelligence calls cost well under a rupee per page, plus infrastructure. The payoff is not that the AP team disappears. It is that most of the team moves from keying to reviewing exceptions, and the reconciliation work that used to sit in a backlog finally gets done.
We learned that the model is the easy part. The validation logic, the review UI, and the discipline of knowing which fields to never trust — that is where the actual work is.