legal document extraction evalsStop Guessing Document AI: Bounding Boxes, Parallel Evals, and Case-Type Routing

Every modern law firm handling high-volume litigation runs into the exact same wall when automating legal intake documents: relying on a single extraction engine to parse every legal instrument is an architectural mistake.

A twenty-page truth-in-lending disclosure requires rigid tabular cell parsing with coordinate stability. An installation contract demands resilience against skewed municipal permit stamps and faded notary seals. A county lien filing demands exact key-value grid capture across numbered form boxes. A disputed sales contract requires agentic reasoning to connect cancellation clauses scattered across separate pages.

No single optical character recognition model or language model excels at every document taxonomy. If you force one provider to handle your entire caseload, you end up paying five times too much for simple forms or waiting two minutes for a page to parse while an interactive review workstation freezes.

At OBE, we solve this through empirical benchmarking and dynamic multi-provider routing. We run automated evaluation suites against gold arbitration records, test spatial unit bounding boxes down to sub-millimeter tolerances, and direct each document type to the provider proven to win on accuracy, speed, and unit economics.

Here is how our evaluation framework operates, the results of our five-document parallel showdown between leading cloud parsing engines, and our routing architecture across five dedicated solar dispute pipelines.

The Benchmark Arena: Five Diverse Legal Instrument Classes

To determine which engine should process specific document types, we built a dedicated parallel evaluation suite. Rather than relying on vendor synthetic marketing claims, our runner executes concurrent, real-time extraction requests across a rigorous corpus of multi-page legal dispute packets.

Our test battery evaluated five representative classes of complex consumer litigation filings:

Sample Classification	Instrument Complexity	Provider A Latency	Provider A Bounding Boxes	Provider B Latency	Provider B Bounding Boxes	50/50 ViewerBox Ready
Sample 1	Multi-Table Loan Disclosure (12 pages)	9.4s	10/10 (100%)	21.2s	7/7 (100%)	Verified Ready
Sample 2	Mixed Grid & Narrative Contract (12 pages)	9.3s	8/8 (100%)	37.2s	6/6 (100%)	Verified Ready
Sample 3	Dense Scan with Skewed Addenda (12 pages)	46.4s	12/12 (100%)	179.8s	7/7 (100%)	Verified Ready
Sample 4	Multi-Column Finance Schedule (12 pages)	9.6s	11/11 (110%)	25.0s	7/7 (100%)	Verified Ready
Sample 5	Bank Arbitration Demand Packet (9 pages)	9.4s	10/10 (100%)	51.1s	5/5 (100%)	Verified Ready

Both engines processed the entire evaluation corpus without a single dropped packet, structural exception, or timeout. The decisive differences emerged in processing latency, unit cost, and document structure strengths.

Spatial Precision and the 50/50 Review Workstation

In legal operations, extracting text alone is insufficient. If an attorney or litigation paralegal cannot click an extracted entity and see the precise bounding box highlighted on the original scanned document, your firm cannot defend the case file.

Our 50/50 split-screen review workstation renders the structured data fields on the left and the raw PDF canvas on the right. When an intake specialist clicks an extracted value, the workstation calculates normalized unit coordinates:

{ left, top, width, height }

The browser converts these unit coordinates into CSS percentage positions:

left: 12.58%, top: 15.72%

This instantly draws a colored SVG bounding box around the source text on the original page canvas.

Spatial Tolerance and Cross-Provider Parity

During our evaluations across the formal arbitration demand class, we compared the physical coordinate outputs of Provider A and Provider B on critical dispute identifiers.

For the primary claimant party name:

• Provider A: Page 1, bounding box { left: 0.1258, top: 0.1572, width: 0.1609, height: 0.0107 }, rendering at CSS left 12.58% and top 15.72%.
• Provider B: Page 1, bounding box { left: 0.1263, top: 0.1565, width: 0.1610, height: 0.0138 }, rendering at CSS left 12.63% and top 15.65%.
• Spatial Variance: Less than 0.05% difference on the horizontal and vertical axes.

For the formal arbitration tribunal reference caption:

• Provider A: Page 1, bounding box { left: 0.7740, top: 0.2407, width: 0.0980, height: 0.0168 }.
• Provider B: Page 1, bounding box { left: 0.7353, top: 0.2437, width: 0.0386, height: 0.0138 }.
• Result: Both engines zeroed in on the exact caption box in the upper right header of the formal demand.

Across our evaluation suite, Provider A delivered 51 valid unit bounding boxes out of 51 citations (100%). Provider B delivered 32 valid unit bounding boxes out of 32 citations (100%). Our coordinate resolution function executed without errors across both providers, confirming complete layout compatibility with our split-screen review interface.

Speed Versus Economics: Comparing Provider A and Provider B

The decision to route a document does not rest solely on precision. It requires balancing end-user review speed against processing expenditure.

1. Speed and Latency Profiling

• Provider A (Deterministic Structural Parser): Averaged 16.8 seconds per document, with a median response time of 9.4 seconds. Because it processes document layouts deterministically without recursive reasoning chains, it delivers lightning-fast turnarounds suited for interactive user workflows.
• Provider B (Agentic Reasoning Parser): Averaged 62.9 seconds per document, with a median of 37.2 seconds. On complex scans, processing reached 179.8 seconds because the agentic engine executes multi-step reflection loops over dense narrative text.

2. Unit Economic Analysis

• Provider A: Approximately $0.015 per page ($0.18 for a standard 12-page legal disclosure packet).
• Provider B: Approximately $0.003 per credit ($0.036 for that same 12-page packet).

Provider B is roughly 5 times less expensive per page than Provider A. For massive document backlogs, historical archives, and overnight batch jobs where latency is secondary, Provider B provides unmatched cost efficiency. When a paralegal is actively waiting on an interactive screen to confirm intake eligibility, Provider A delivers the 9-second response time required for real-time operations.

Multi-Provider Routing Across Five Solar Dispute Pipelines

Rather than settling on a rigid compromise, the OBE platform configures per-document extraction routing. We assign each legal document type to the specific engine engineered to dominate that document topology:

Pipeline 1: Solar Loan Finance Agreements (solar.finance_contract)

• Target Entities: lender_name, principal_amount, apr, monthly_payment, term_months, dealer_fee
• Assigned Route: Provider A (Primary)
• Engineering Justification: Finance agreements are centered around rigid Truth-in-Lending Act (TILA) disclosure tables. Provider A excels at multi-column financial grids, preventing coordinate distortion across bordered table cells and nested interest rate calculations.

Pipeline 2: Solar Purchase and Sales Agreements (solar.solar_agreement)

• Target Entities: installer_company_name, homeowner_name, total_contract_price, system_size_kw, cancellation_deadline
• Assigned Route: Provider B (Agentic)
• Engineering Justification: Solar purchase agreements scatter critical terms across dense, multi-page narrative blocks, addenda, and handwritten change orders. Provider B uses multi-step reasoning to synthesize disparate clauses and confirm statutory three-day cancellation rights at one-fifth the processing cost.

Pipeline 3: Installation and Workmanship Agreements (solar.installation_agreement)

• Target Entities: contractor_name, installer_company_name, permit_date, system_completion_date, workmanship_warranty_years
• Assigned Route: Provider A (Primary)
• Engineering Justification: Installation agreements frequently feature scanned municipal permit stamps, structural engineering stamps, and degraded signatures. Provider A delivers superior character retention on skewed or low-contrast stamped sections.

Pipeline 4: County UCC-1 Financing Statements and Fixture Filings (solar.ucc_1_lien)

• Target Entities: debtor_name, secured_party_name, filing_number, filing_date, collateral_description
• Assigned Route: Provider A (Primary)
• Engineering Justification: The standard UCC-1 Form 1 follows a fixed, numbered box grid (boxes 1a, 1b, 2, and 4 collateral). Provider A maps form-field bounding boxes in approximately nine seconds with zero horizontal coordinate drift.

Pipeline 5: Arbitration Demands and Dispute Invoices (solar.arbitration_demand_invoice)

• Target Entities: case_caption, claimant_name, respondent_name, arbitration_reference_number, invoice_amount, invoice_number, service_date
• Assigned Route: Configurable by Firm Workflow
• Engineering Justification: Both providers achieved 100% bounding box integrity on arbitration pleadings and fee invoices. Law firms running interactive intake desks select Provider A for sub-ten-second updates. Litigation practices conducting high-volume portfolio acquisitions select Provider B to minimize processing costs.

Book a demo →

Modern legal intake demands precision at the pixel level. By running continuous evaluation benchmarks and routing each document to its specialized engine, your firm gains sub-second workstation speed and unassailable coordinate citations.

End-to-End Citation Provenance: From Raw Pixels to Court-Admissible Proof

Parsing text and rendering SVG boxes on an internal canvas solves only half the legal battle. If a defense firm challenges an arbitration demand, your litigation team cannot show up with unanchored strings. They need complete, verifiable citation provenance from the moment a file enters your cloud datalake to the second an attorney signs off on the statement of claim.

In the OBE architecture, citation provenance is engineered as a zero-trust cryptographic chain:

1. Hierarchical Document Trees and Bates-First Labeling

When a document enters our intake pipeline, OBE does not flatten the file into arbitrary text chunks. Our pipeline constructs a hierarchical document tree structure (documentTrees and documentTreeNodes).

Every section, paragraph, table, and header receives a deterministic structural identifier (such as sec_0001.sub_0002.para_0003) alongside its physical page range and exact bounding box coordinates ({ x0, y0, x1, y1 }). When documents undergo formal production, our indexing engine stamps canonical Bates numbering directly onto each tree node (DISPUTE-000123). When an intake specialist inspects evidence in ParalegalView or CaseData, every citation leads with verified Bates stamping rather than ephemeral file paths.

2. Cryptographic Checksums and Hard Drift Guards

Legal filings frequently evolve over months of discovery. If a defense vendor quietly swaps a contract version, re-uploads a disclosure with altered terms, or replaces a scanned addendum, unanchored intake databases fail silently.

OBE stops this vulnerability through immutable checksum tracking. Every source file registers a cryptographic SHA-256 hash (storageFiles.checksum). At the moment of extraction, this hash snapshots directly into our review state machine:

• If a source file changes after initial parsing, OBE detects the hash discrepancy instantly.
• The platform immediately throws a source_document_changed exception, halts automated approvals, and raises a prominent drift alert across the reviewer workstation: "This document changed since extraction - re-verify before relying on this citation."
• Reviewers are prevented from approving drifted records without an explicit, audit-logged acknowledgment.

3. Provenance Tiers and Sworn Review State Machines

Every extracted field in OBE carries an explicit provenance tier that declares its evidentiary fidelity:

• Tier 1 (Exact Region / Masked Crop): Direct bounding box citation linked to a discrete structural document tree node.
• Tier 2 (Page-Level Location): Sourced to an authenticated page container when narrative clauses span multi-column layouts.
• Tier 3 (Synthesized Fact / Entity Reference): Inferred through cross-document reconciliation across multiple exhibits.

These tiers feed into an eight-stage review state machine: pending -> assigned -> approved -> flagged -> corrected -> consensus -> adjudicated -> signed

Multiple human labelers review sensitive fields in isolation. Disagreements route to an adjudication queue where managing attorneys select the winning value against the original source evidence. When the file clears review, an attorney completes a sworn electronic sign-off, affirming under professional ethical rules that each cleared field was verified against its cited source document.

4. The Exportable Provenance Certificate

Before a claim package files with JAMS, the American Arbitration Association, or a state court, OBE synthesizes the entire evidentiary chain into an exportable Provenance Certificate.

This PDF artifact records the complete chain of custody:

• The case caption, filing number, and timestamped generation manifest.
• An itemized register of every case entity, its verified value, and its Bates-stamped source reference.
• The SHA-256 cryptographic checksum of the underlying storage file.
• The identity, role, and timestamp of every paralegal who reviewed the field and the attorney who executed the sworn affirmation.

If opposing counsel alleges that automated software fabricated case facts, your firm hands them an unimpeachable, cryptographically sealed verification certificate.

Building Evidentiary Integrity Into Intake

Document extraction in civil litigation is not about generating loose summaries. It is about producing verified case packages that withstand intense judicial scrutiny and aggressive defense challenges.

When your intake system extracts an annual percentage rate or a contract execution date, that field must remain bound to its exact location on the original contract. If a defense firm claims a homeowner agreed to an arbitration waiver or signed away their warranty rights, your firm can instantly display the unit bounding box in your case dashboard.

By combining empirical provider evaluations with automated document-specific routing, OBE gives plaintiff law firms an industrial data pipeline:

• Zero Coordinate Distortion: Every extracted entity maps to normalized coordinates with under 0.05% spatial drift.
• Optimized Unit Economics: High-volume narrative contracts route to cost-efficient agentic engines, saving up to 80% on document processing.
• Instant Paralegal Review: Complex finance grids and numbered liens parse in under ten seconds, eliminating workstation delays.
• Full Provenance Defense: Bates-stamped document trees, cryptographic drift guards, and exportable verification certificates protect your firm against defense challenges.

Book a demo →

To review our extraction evaluation harness, inspect our 50/50 review workstation, or schedule an architecture consultation with founder Tim Ottowitz, visit our demo scheduling portal or explore our legal intake software platform.