Agentic Document Extraction: Turn Invoices into Data

Agentic document extraction is a method in which an AI model plans the job, calls layout, OCR, table and form tools, checks the results against rules and retries the fields that fail. Invoices and contracts come out as clean rows and fields: JSON, a spreadsheet row, a bill in QuickBooks or Xero, or a record in a CRM. Nobody writes a template for each vendor.

Extraction is the step between a document arriving and its data being used. Approval routing, purchase order matching and payment come after it and follow different rules.

Viktor is the AI employee who lives in Slack and Microsoft Teams, connects to 9,800+ tools, and does the work. For document extraction, a finance or operations lead describes the job in plain language. Viktor returns the structured file or the updated record, and he waits for approval before writing to a finance system.

What agentic document extraction is

LandingAI’s Agentic Document Extraction (ADE) is the clearest named product in the category. Its Parse step converts a document into structured Markdown and hierarchical JSON with page and coordinate references. Its Extract step then pulls specific fields with a JSON schema.

Research systems spell out the loop in more detail. AgenticIE wraps deterministic utilities such as OCR and PDF parsing, plus model-powered checks such as verification, behind one callable interface. Its planner first tests whether a document is scanned, then calls OCR or direct text extraction, and can later call a verification step to check that the output is complete and grounded in the source text.

Andrew Ng described the reasoning in a DeepLearning.AI item on his letter: “Humans don’t just glance at a document and reach a conclusion on that basis. Instead, they iteratively examine different parts of the document to pull out information piece by piece. An agentic workflow can do the same.”

The output usually comes in three shapes:

• Markdown: A readable version of the document for people to check, which LandingAI ADE and Docling both export.
• JSON: Typed fields that a spreadsheet, accounting system or CRM can take, such as Docling’s lossless JSON or the LlamaParse JSON and XLSX outputs.
• Key-value pairs: A label and its value, such as the keyValuePairs feature in Azure Document Intelligence’s layout model.

Single-pass OCR differs on two counts. It returns text in a default order with no check on whether the result makes sense, and a single model prompt returns one answer with no step that re-reads a failed field.

The “agentic” label is also applied loosely. A Towards Data Science essay notes that many systems sold this way have “No LLM at the dispatch layer decides what to run next.” Ask any vendor two questions: does a model decide the next step, and are failed fields re-requested?

Why OCR and templates break on invoices and contracts

Tesseract’s own documentation states that the engine has a known problem recognizing text and data from tables without custom segmentation or layout analysis. Invoices are mostly tables. The same page says line segmentation quality drops sharply when a page is too skewed.

The failure modes repeat across invoices and contracts:

• Tables and merged cells: Early row and column splitting methods failed on cells spanning several rows or columns, and invoice line-item tables run from two rows to several pages.
• Multi-column layouts and reading order: Most OCR engines arrange text top to bottom and left to right, which the LayoutReader authors call not optimal for multi-column forms and invoices, and in multi-column invoices headers such as Amount or Description may not align with their values.
• Checkboxes: A transfusion-report study tested 24 clinical forms with faint or ambiguous marks: OCR read 16 correctly (66.7%) and a vision-language model method read 21 (87.5%).
• Mixed PDFs: Routing a PDF that mixes digital and scanned pages entirely through text extraction silently drops the scanned pages.
• Vendor layout changes: OCR-plus-rule systems built on fixed zones per template are brittle and do not generalize to unseen invoices, and in one electricity-invoice study precision dropped to 67.20% on unseen templates.
• Contract strikethroughs: In a RAG robustness paper, a clause with struck-through terms came out of OCR as “ninety (90) sixty (60) days” and “$5,000 $2,000 per day”.

A practitioner writing in Towards AI described a major vendor updating its invoice template: field positions shifted, the date format changed from MM/DD/YYYY to YYYY-MM-DD, and the model trained on the old format began extracting the wrong fields. If a team maintains one template per vendor today, count how many vendors changed layout in the last year; that is the maintenance load an agentic approach removes.

How the extraction loop works

A single invoice PDF shows the loop most clearly. The input is the file. The transformation is a sequence of tool calls with checks between them. The output is a validated record with a pointer back to where each value sits on the page.

The sequence runs in this order:

• Plan: The model checks the file type and whether each page is scanned or digital, as in the AgenticIE modality check.
• Detect layout: A layout model finds headings, tables, form fields and reading order before any text is read.
• Extract: OCR or a vision-language model reads each region, and the extractor fills a JSON schema.
• Validate: Rules check the result. POLARIS lists schema completeness, cross-field arithmetic, date logic and external consistency.
• Retry: Only failed fields go back. POLARIS issues localized repairs in which only implicated fields are re-requested, inside a bounded loop.
• Output: The record leaves with its grounding data, or goes to a person if it still fails.

Vision-language models (VLMs) read a page as an image rather than as OCR text. The Gemini API counts 258 tokens per document page, and Claude’s PDF support converts each page to an image in addition to its text.

Visual grounding ties every extracted value to a place on the page. LandingAI’s Extract grounds each value to a character range in the parsed Markdown, which its Ground API maps to a page number and a bounding box. LlamaParse Extract citations carry the page number, the matching text and box coordinates.

General-purpose VLMs are less precise here. Anthropic’s vision documentation says Claude’s coordinate outputs are approximate and should be verified before anyone relies on them. When grounding matters for audit, require a parser that returns page and box coordinates for every field.

Before you start

A pilot needs four things in place before the first document is processed:

• A sample set: Real invoices and contracts from the last few months, pulled from the inbox and shared drive, not a cleaned-up folder.
• A field list: The exact fields each document type must return, written down before any tool is chosen.
• A destination: The spreadsheet, QuickBooks, Xero or CRM that receives the data, plus its rules; a QuickBooks Online Bill requires a VendorRef and at least one line, and its DocNumber is capped at 21 characters.
• Access: Read access to the inbox or folder where documents arrive, and write access to the destination, granted with approval rules.

Xero adds its own constraints. Its CSV import creates invoices as drafts and requires AccountCode and TaxType to match Xero exactly, or the line will not import. Pull the chart of accounts and tax codes from the destination before defining fields.

How to get invoices and contracts into structured data: step by step

An owner gets reliable structured data from invoices and contracts in eight steps: define the fields, collect a representative sample, choose who runs the job, set validation rules, pilot against a hand-checked set, route exceptions to people, send the output to its destination, and make the run recurring. Each step below settles one decision.

Step 1: Decide which fields each document type must return

Invoice fields map directly to the accounting system: vendor, invoice number, invoice date, due date, currency, line items (description, quantity, unit price, line total), subtotal, tax and total. Xero bills are created as ACCPAY invoices, and Xero calculates tax per line, rounded to two decimals, so capture tax at line level where the document shows it.

Contract fields come from the CUAD labeling handbook: Parties, Effective Date, Term, Expiration Date, Renewal Term, Notice Period to Terminate Renewal and Governing Law. CUAD has no contract value label; Ironclad’s renewals view defines Contract Value as the total cost of the contract over its full initial term.

A simple invoice schema looks like this:

A contract schema follows the same pattern:

Nullable fields are deliberate. Google’s structured output documentation says nullable properties let the model return null when it lacks context, which helps reduce hallucinations. Mark a field required only if a missing value should stop the record.

Step 2: Choose a sample that reflects the documents you actually receive

A practitioner write-up recommends recording correct values field by field for 50 to 100 documents. Mix vendors, digital PDFs, scans, phone photos, multi-page files, contracts with amendments and anything with handwritten notes.

Testing only on clean digital PDFs gives a false reading. OmniDocBench, the common parsing benchmark, relies mostly on born-digital images, as the Real5-OmniDocBench authors point out. On real-world captures in Wild_OmniDocBench, MinerU2.5 fell from 90.67 to 70.91 overall.

Handwriting needs its own slice of the sample. In the METATR benchmark, the best model’s character error rate was 17.6% on printed documents and 28.5% on handwritten ones. If handwritten approvals or notes carry data the team needs, include at least a handful in the sample.

Step 3: Decide whether to hand off, buy a parsing API or build

Handing it off to Viktor suits a team with no developer to spare. The request reads like an instruction to a new hire: every morning, read the new invoices in the AP folder, pull vendor, number, dates, line items, tax and total, check that the lines add up, and add them to the bills sheet.

He plans the work, runs it on his own cloud computer and returns the file or the updated record in the thread.

That is what an AI employee is: a hire who owns a recurring job across the tools a company already uses. Viktor reads the PDFs, spreadsheets and images people send him and produces Excel files with formulas.

If the team already pays for a parsing API, any tool with an API can become a custom integration, and Viktor can usually build the missing connection himself.

Viktor is free to start with up to $100 in credits and no credit card. The Team plan is $100 a month for 40,000 credits shared across the whole workspace, with no per-seat fee and unused credits rolling over (pricing checked October 2026).

Start free with up to $100 in credits, no credit card →

Buying a parsing API suits a team with one developer who can call it. LandingAI ADE pairs Parse and Extract with a Ground API that maps each extracted field back to its page location. LlamaParse offers Parse tiers from Fast to Agentic Plus, and LlamaExtract returns per-field citations when cite_sources is enabled.

Building on an open-source framework needs a Python developer:

• Docling: Its PyPI page lists version 2.149.0 (released Nov 8, 2026), which parses PDF, DOCX, XLSX, images and more, runs locally including air-gapped setups, and installs with pip install docling on Python 3.10 or higher; schema-guided field extraction lives in the separate docling-agent project.
• smolagents: Version 1.26.0 (released May 29, 2026) runs model-driven programs that write and execute code, lists no document-extraction tool among its built-in tools, and its PyPI page warns that the built-in LocalPythonExecutor is not a security sandbox.
• Strands Agents: Its PyPI page lists version 1.64.0 (released Nov 4, 2026), licensed under Apache 2.0 and defaulting to Amazon Bedrock, which needs AWS credentials; its PDF example is a skill the user writes, not a built-in extractor.

If nobody on the team writes Python, the frameworks are out. If someone does but has no time to maintain a pipeline, buy the parser or hand off the job.

Step 4: Set the validation rules that decide what passes

Rules such as line totals summing to the subtotal and a due date falling on or after the invoice date are where accounting knowledge enters the system. Each rule should be checkable without judgment:

• Arithmetic: Line totals sum to the subtotal, consistent with the sum checks in an invoice extraction study. An invoice pipeline guide also lists checks that subtotal plus tax equals the total and quantity times unit price equals each line total.
• Dates: Invoice and due dates parse into one format, and the due date falls on or after the invoice date.
• Required fields: Vendor, line items and total are present, and a missing one stops the record.
• Destination rules: No duplicate invoice number for the same vendor, since QuickBooks Online throws an error on a duplicate DocNumber unless a special parameter is set.
• Confidence thresholds: Microsoft’s Content Understanding guidance gives example thresholds of 0.90 for critical fields such as TotalAmount, 0.80 for important fields such as VendorName and 0.70 for non-critical fields.
• Flags: Any field below its threshold is flagged for review rather than written.

Treat model confidence as one signal, not the decision. One paper found token probabilities saturate above 0.999 with structured JSON output, so arithmetic and required-field checks carry more weight than a score.

With Viktor, the rules go into a skill: a written procedure he follows on every run, which the team can open and edit. Write the arithmetic and required-field rules first, then add confidence thresholds once the pilot shows where errors cluster.

Step 5: Run a pilot and decide if accuracy is good enough

Field-level accuracy against the hand-checked set is the measure that matters. An invoice extraction study scores header fields with exact match plus a relaxed match that allows tolerance windows for amounts and dates, and scores line items by whether values land in the right row. The data-alchemy write-up warns against averaging header and line fields into one figure.

Run single-pass and multi-step extraction on the same tables, forms and scans. Published comparisons show a gap, though the cleanest ones come from the vendor whose tiers are being compared:

• ParseBench: LlamaParse’s single-pass Cost Effective configuration scored 71.89% overall and its multi-step Agentic configuration 84.88%, in a benchmark authored by LlamaIndex.
• ExtractBench: The ExtractBench paper reports that LlamaExtract’s Cost-Effective tier reached 86.8% F1 at 1.0 ¢ per page, Agentic 89.5% at 3.1 ¢ per page and Agentic Plus 95.6% at 8.1 ¢ per page.
• KYC pipeline study: The authors’ ACL 2026 paper reports that a multistage OCR, retrieval and VLM pipeline on about 3,000 scanned pages improved field-level accuracy by up to 31.9 percentage points over direct PDF-to-VLM baselines.

Set a pass bar per field before the pilot starts, for example exact match on invoice number and total. If the multi-step run does not beat single pass on the fields that matter, keep the cheaper method.

Step 6: Decide which fields go to human review

AWS’s Augmented AI tutorial shows the pattern: send a review when a critical key such as Mail Address scores below 99, or any other key-value pair below 90. Its JSON examples also send 5% of high-confidence results to review as a random audit.

Thresholds come from the pilot, not from a default. Microsoft’s transparency note tells customers to pilot their use case to estimate the threshold for straight-through processing versus human review.

Viktor is the AI employee who lives in Slack and Microsoft Teams, connects to 9,800+ tools, and does the work, and he waits for approval before he changes an account. Flagged fields come back in the thread with the reason, and before he writes a bill into QuickBooks or Xero he posts approval cards in Slack or Microsoft Teams with Approve and Reject buttons; a rejected action never runs. Viktor waits for explicit approval for money moves, code pushes and customer emails, and admins can set any other integration tool to “needs approval” or “forbidden” (viktor.com/security).

Set the accounting integration to “needs approval” for the first month. Relax it only for vendors whose bills passed review without edits.

Step 7: Choose where the output goes

Choose the output format to match the destination. These are the common targets:

• Spreadsheet: An Excel file with formulas, or a Google Sheet, for teams that review before posting.
• QuickBooks Online: Bills through the API, or invoices through CSV import, which accepts up to 100 invoices at a time and, in the US version, not when sales tax is set up in the account.
• Xero: Bills as ACCPAY invoices, where line amounts default to tax exclusive if LineAmountTypes is not specified.
• CRM: Contract value and renewal dates on the account record.
• Contract repository: Fields such as Ironclad’s Renewal Opt Out Date, the deadline to give notice that prevents auto-renewal.
• Contract search: Chunked output for retrieval, such as Docling’s Chunks (JSONL) format.

Check the integrations directory to confirm the destination connects before the pilot ends. Approval chains, purchase order matching and payment runs belong to the full accounts payable workflow, covered in the separate guide on invoice processing automation.

Step 8: Decide what starts each run

Three starting points are available for a run. A message in a channel such as #invoices starts the job when someone drops in a PDF. A scheduled task checks a Google Drive folder every morning. A webhook from another tool starts the job when a file lands there. Viktor also has his own inbox and receives email from approved senders.

Call these tasks, not automations: the team describes the job once in plain English, and Viktor runs it on the schedule. For a messy process, record the screen once with narration; Viktor turns the recording into a written skill the team can correct, and if the job repeats he proposes a scheduled task that stays paused until someone approves it.

Viktor works in Slack, in Microsoft Teams (generally available since 18 June 2026, with scheduled tasks and triggers on messages and meetings) and in the web app at app.viktor.com. Pick the trigger that matches where documents already arrive.

Hand over the invoice inbox today, free with up to $100 in credits, no credit card →

When a multi-step approach is worth the cost

• A datasheet extraction study measured the trade-off. At list prices, the agentic tool layer cost 1.2 times the single-pass baseline with Claude and 1.8 times with GPT-5.1, and baseline and agentic runs agreed on fidelity in 68 of 75 cases. Every document in its main corpus fit one context window, so single pass was the right call there.
• The picture flipped on a 397-page datasheet. The native PDF single-pass request was rejected, because the Claude gateway caps PDFs at 100 pages, while the agentic run extracted all five target parameters on each of three models in 83 to 139 seconds. The authors recommend defaulting to single pass and switching only when a document exceeds the context window or a page cap.
• Multi-step designs also add latency. A financial document study found a hierarchical multi-agent setup raised F1 from 0.897 to 0.921 over a sequential one, while end-to-end time rose from 34.2 to 41.8 seconds per document and cost per document rose too.
• Token use grows with every page and every retry. Anthropic’s PDF documentation puts each page at 1,500 to 3,000 text tokens, plus image tokens for the page render.

Skip the multi-step approach for uniform digital forms: the same five vendors, the same layouts, text already embedded in the PDF. On an OpenAI community thread about a 3-page structured PDF, a reply argued that putting the extracted text into one prompt is fine and cheaper in tokens. Use multi-step for scans, mixed vendors, long contracts and anything that failed the pilot in single pass.

When Viktor runs the job, the Activity view shows credit use by person and task, each scheduled task shows its credit cost and time saved, and spend limits with a monthly top-up cap keep usage bounded. See how shared workspace credits work → (pricing checked October 2026)

Troubleshooting common extraction failures

A loan agreement in one hallucination study showed the most expensive failure: a base model returned “January 1, 2024” for an Effective Date the document stated as March 15, 2024. Each common failure has one safeguard:

• Hallucinated fields: Make every non-critical field nullable with a description of when null applies, so the model returns null instead of inventing a value.
• Retry loops: Cap retries in code; in the Instructor library, max_retries=2 means at most three SDK calls before the record goes to review.
• Wrong reading order: Run a layout parser that detects reading order before extraction, such as Docling’s page layout and reading order analysis.
• Split tables: Enforce the line-items-sum-to-total check, which catches rows lost when VLM-based OCR breaks tables across page boundaries.
• Poor scans: Reject pages below 200 dpi and request a rescan, following Google Document AI’s guidance that 300 dpi and higher generally gives the best results.

Log every failure with its document and field during the first month. The pattern shows which safeguard to add next.

Security and privacy for sensitive documents

Invoices and contracts carry what OWASP LLM02:2025 lists as sensitive: financial details, confidential business data and legal documents. The first decision is whether documents leave the company’s infrastructure at all.

Self-hosted extraction keeps them inside. Docling lists local execution for sensitive data and air-gapped environments, LlamaParse offers self-hosted deployment on Enterprise plans, and LandingAI lists VPC and on-premises deployment as Enterprise-only. Self-hosting needs people to run the infrastructure.

Third-party APIs keep documents for set periods. LlamaParse caches parsed documents for 48 hours unless do_not_cache=True is set. Azure Document Intelligence stores input and results for 24 hours after analysis. Google Document AI processes online requests in memory without persisting them, with a one-day failsafe for batch jobs.

NIST’s AI RMF Generative AI Profile recommends contracts that specify content ownership, usage rights and security requirements. Before a pilot, get the vendor’s retention period, training policy and subprocessor list in writing.

As of October 2026, viktor.com/security lists Viktor as SOC 2 Type 1, with SOC 2 Type 2 and ISO 27001 in progress. Data is encrypted with TLS 1.2+ in transit and AES-256 at rest, each workspace is isolated, and customer data is never used to train models, by Viktor or his model providers. API keys and tokens are added at run time by a gateway, so the model never reads credentials, and data is US-hosted by default.

Review Viktor’s security details, data handling and approval controls → For a company-wide rollout, talk to the Viktor team about onboarding, security review and spend controls →

FAQ

Invoices and contracts need different field checks, and document quality affects the review needed before extracted values reach a destination.

How does agentic document extraction differ from OCR or a single prompt?

OCR converts pixels to text, and a single prompt returns one answer; agentic document extraction adds planning, tool calls, validation and targeted retries. POLARIS, a back-office research system, argues that untyped tool calls and open-ended retries make pipelines brittle with unclear provenance, which is why the retry loop is bounded and field-specific.

How accurate is it?

Accuracy depends on the documents, so the only trustworthy figure comes from the team’s own hand-checked set. Published numbers vary widely: in the DocILE benchmark, a RoBERTa baseline scored line-item F1 of 0.686, while vendor benchmarks report mid-90s F1 for their top tiers.

Does it handle handwriting and poor scans?

Yes, with lower accuracy than on clean print. In a handwritten forms benchmark, GPT 5.4 had the lowest hallucination rate at 6%, and Anthropic’s vision documentation says Claude may make mistakes on low-quality, rotated or very small images under 200 pixels.

Do contracts need a different setup from invoices?

Yes, because near-plain-text contracts encode little in their layout while layout-rich invoices depend on position, as an ACL 2026 paper notes. Contracts need clause-level fields such as notice periods, which a WorldCC procurement report puts at typically 30 to 90 days and sometimes 180 days for enterprise agreements.

What does extraction cost per document?

Cloud parsers price per page or per document, pricing checked November 2026. Azure Document Intelligence’s prebuilt models, including Invoice and Contract, cost $10 per 1,000 pages on pay as you go.

AWS Textract’s Analyze Expense costs $10 per 1,000 pages for the first million pages per month in US East (Ohio).

Google Document AI’s Invoice parser costs $0.10 per count, where one count covers up to 10 pages.

LandingAI calibrated its DPT-3 Pro parse at roughly 1.5 credits per page for a typical business document with 4,000 characters on the page on the Standard tier, with one credit equal to one US cent.

Do you need a developer?

No, not if the job is handed to an AI employee or a hosted tool that connects to the destination; frameworks such as Docling, smolagents and Strands Agents do need one. The Viktor Academy guides walk through connecting tools and handing over a first job.

Hire your first AI employee: start free with up to $100 in credits, no credit card →