LightOnOCR 3: extract a document, then check every important field

What the documentation establishes

LightOnOCR 3 documents plain transcription and grounding with normalized layout regions. Those regions do not establish transcription accuracy. Source ↗

The client repository documents local serving, separate plain/grounding runs and page-level output files. Source ↗

The walkthrough below is an original worked exercise. We have not independently tested this product’s performance.

1. Build a page that can expose a believable mistake

Create a one-page document headed “Juniper Supplies — TRAIN-0042”. Add the two rows below, a total of 46.50, and a separate footer note saying “Last month’s total: 46.80”. Export it as receipt.pdf. The similar numbers make it possible to detect whether the extraction confused current values with nearby context. Use fictional data throughout this first trial.

Save an answer key before running any model. It should contain the document ID, both item descriptions, quantities, unit prices, current total and page number. Keep that answer key outside the model prompt. Make a second copy with the table slightly rotated or a faint total; change one property at a time so a new error has an interpretable cause. Do not treat either synthetic page as a representative production benchmark.

Juniper Supplies
Document: TRAIN-0042

Item       Quantity   Unit price
Notebook   3          12.50
Marker     2           4.50

Current total: 46.50

Note: Last month’s total was 46.80.

2. Use the documented client and preserve its environment

The project’s quick start uses the 1B model served on port 8010. Follow the repository’s supported environment and pinned dependencies; this is an optional setup path, not a claim that it runs on every laptop. Review the repository and model terms before installing. Record the resolved repository commit, model revision and hardware in your run notes.

The commands below adapt the documented quick start to our receipt. Run the server in one terminal. In a second terminal, activate the same environment and run the client commands. The vLLM extra currently pins compatible dependencies; arbitrary upgrades can change behavior. If you already have an approved compatible server, use the project’s documented client configuration instead of starting another.

git clone https://github.com/lightonai/LightOnOCR.git
cd LightOnOCR
uv sync --extra vllm
source .venv/bin/activate
vllm serve lightonai/LightOnOCR-3-1B \
  --limit-mm-per-prompt '{"image": 1}' --max-model-len 16384 \
  --mm-processor-cache-gb 0 --no-enable-prefix-caching --port 8010

cd LightOnOCR
source .venv/bin/activate
lightonocr /absolute/path/receipt.pdf --name receipt-plain
lightonocr /absolute/path/receipt.pdf --mode grounding --name receipt-grounded
lightonocr viewer

3. Compare the rendered page with the transcription

The client keeps the submitted page image, Markdown, and grounded block data in its output folder. Inspect the image actually sent to the model, not only your original PDF: rendering or rescaling can make a previously clear digit unreadable. Compare the two runs in the viewer and record whether reading order, row alignment and the footer survived.

The documented plain request contains the image; the grounding request adds the word grounding. Extra business instructions are outside the documented OCR prompt format. First obtain the transcription, then perform field mapping in a separate step. Do not ask this transcription model to produce your company’s custom JSON and assume the same behavior.

For each accepted field, record a page and supporting region. The model card uses normalized coordinates from 0 to 1000. A region is a pointer to inspect, not proof that the text inside is correct. Confirm the total points to the current-total line and the two prices remain attached to the correct descriptions. Mark unreadable values unknown rather than filling them from the arithmetic answer key.

4. Map into a small review record

Write review.json using the adapter shape below. This is our own handoff format, not LightOnOCR’s native block schema. In a real integration, your adapter must deliberately map the parsed blocks into these fields. For this exercise, transcribe the OCR result manually and copy the supporting region only after inspecting the page.

The illustrated box is invented and must be replaced with the observed coordinates from your actual layout. Preserve the raw Markdown and block JSON alongside this record so a reviewer can revisit a disputed interpretation. Use strings for decimal amounts to avoid introducing binary floating-point rounding into a straightforward total check.

{
  "document_id": "TRAIN-0042",
  "page": 1,
  "items": [
    {
      "description": "Notebook",
      "quantity": 3,
      "unit_price": "12.50"
    },
    {
      "description": "Marker",
      "quantity": 2,
      "unit_price": "4.50"
    }
  ],
  "total": "46.50",
  "total_bbox": [
    700,
    650,
    900,
    720
  ]
}

5. Check arithmetic, then prove the check can reject errors

Save the following as check_receipt.py and run python3 check_receipt.py review.json. A passing result only establishes the listed structural and arithmetic conditions. Two wrong values can still agree with each other; you must also compare the accepted fields with the held-out page answer key. We ran this original validator against the illustrated record and deliberately incorrect copies; it accepted the fixture and rejected the bad total and reversed box.

Make a disposable copy with total changed to 46.80; the arithmetic check must fail. Restore the total and reverse the horizontal box edges; the coordinate check must fail. Finally swap the two item descriptions without changing their numbers. This intentionally passes the program but should fail your visual review. That last case demonstrates why a syntactic validator cannot replace field-to-page verification.

import json, sys
from decimal import Decimal

def check(record):
    if record['document_id'] != 'TRAIN-0042' or record['page'] != 1:
        raise ValueError('Wrong fixture or page')
    items = record['items']
    if len(items) != 2:
        raise ValueError('Expected two line items; compare the page')
    computed = Decimal('0')
    for item in items:
        qty = item['quantity']
        if type(qty) is not int or qty <= 0:
            raise ValueError('Quantity must be a positive integer')
        price = Decimal(item['unit_price'])
        if not price.is_finite() or price < 0:
            raise ValueError('Invalid price')
        computed += qty * price
    total = Decimal(record['total'])
    if not total.is_finite() or computed != total:
        raise ValueError('Line items disagree with extracted total')
    bbox = record['total_bbox']
    if len(bbox) != 4 or any(type(v) is not int for v in bbox):
        raise ValueError('Expected four integer coordinates')
    x1, y1, x2, y2 = bbox
    if not (0 <= x1 < x2 <= 1000 and 0 <= y1 < y2 <= 1000):
        raise ValueError('Invalid normalized box')
    print('Structural and arithmetic checks passed; visual review still required')

if __name__ == '__main__':
    with open(sys.argv[1], encoding='utf-8') as f:
        check(json.load(f))

6. Diagnose failures before increasing the batch size

If the server does not start, resolve environment and memory requirements first; a Python traceback is not an OCR-quality result. If the client cannot connect, check the configured endpoint and whether your server is still running. If output repeats or becomes nonsensical, return to the repository’s pinned serving versions before changing the document or adding prompt instructions.

If the image looks clear but a field is wrong, classify the failure: missing text, incorrect character, wrong row association, or incorrect reading order. If the submitted image itself is poor, improve the rendering or source scan and record that intervention. Keep the failing page in a small regression collection. Do not quietly remove difficult examples to make the next score look better.

Before real use, test several representative pages with known answers and hold some out from tuning. Report field accuracy, unresolved-field rate and review time separately. A fast transcription that doubles manual correction can be the worse workflow. This exercise makes no comparative speed or accuracy claim about LightOnOCR.

7. Deliver evidence and a clear next action

Package the input PDF, the exact submitted image, raw transcription, grounding data, review record, validator output and a short unresolved-items note. Record model/repository revisions, date and any human correction. Keep original output intact; a corrected field should carry both values and a reason, so later readers can distinguish machine output from reviewed data.

Use an explicit acceptance decision: accepted for this task, requires review, or unsupported. For example, an unreadable total means requires review even if the line-item arithmetic is internally consistent. Expand to another document type only after the first has repeatable checks. Tables, handwriting, multi-column papers and charts should each get their own fixtures and failure analysis.

Input and submitted-image hashes: <record>
Model and repository revisions: <record>
Raw output location: <record>
Correct fields / reviewed fields: <count>
Unresolved fields: <list>
Arithmetic check: <pass/fail>
Visual region and row check: <pass/fail>
Negative controls: <results>
Human corrections: <before, after, evidence>
Decision and next owner: <record>

Guide history

Revisions preserve the original instructions. A source recheck alone does not create a new revision.

Revision 1 · New worked document fixture, documented serving path, original tested validator, negative controls and evidence handoff. Model inference remains untested.

Continue learning