feat: Layout-driven OCR pipeline with configurable OCR modes and refactoring of the OCR GT dir structures

[nikos-livathinos]

Reorders the standard PDF pipeline to be layout-driven and introduces a configurable OcrMode that controls which document regions are fed to the OCR engine. Layout prediction now runs before OCR, so OCR can be targeted at the clusters that actually need it, and layout post-processing is split out into its own pipeline stage.

• Refactor the code to decouple the layout model and the layout post-processing as separate pipeline stages.
• Redesign the StandardPdfPipeline to run: Preprocess -> Layout -> OCR -> Layout post-processing -> Table -> Assembly.
• Introduce OcrMode with the options:
  • FULL_PAGE: Run OCR on the full page. No layout clusters or PDF cells are taken into account.
  • LAYOUT_REGIONS: All detected bounding boxes from the layout model are fed as input to the OCR. No PDF cells are taken into account.
  • PDF_AWARE_LAYOUT_REGIONS: Start with all layout detections and eliminate the ones that overlap with PDF cells that contain exclusively text. If there is any non-text overlapping PDF cell, the layout bbox is not eliminated. Non-overlapping layout bboxes are kept. All non eliminated bboxes are fed as input to the OCR.
  • DEFAULT: Placeholder value. Currently hard-wired to PDF_AWARE_LAYOUT_REGIONS.
• Refactor the BaseOcrModel.post_process_cells() to introduce the optional parameter priority that controls how the detected OCR cells and the pre-existing PDF cells are merged into the final page cells:
  • PDF_FIRST: PDF_FIRST: The OCR cells are used only if they do not overlap with any PDF cell.
  • OCR_FIRST: The opposite. The PDF cells are used only if they do not overlap with any OCR cell.
  • When priority is not provided (None), it is auto-selected from the OcrMode: OCR_FIRST for LAYOUT_REGIONS, and PDF_FIRST for PDF_AWARE_LAYOUT_REGIONS.
  • The priority parameter is NOT exposed to the OcrOptions.
• Remove the OcrOptions.bitmap_area_threshold
• Deprecate the OcrOptions.force_full_page_ocr parameter. Now it is superseded by OcrMode.FULL_PAGE
• Expose the OcrMode to the CLI via the --ocr-mode parameter:
  • The old --force-ocr is still supported but deprecated.
  • In case both --ocr-mode and --force-ocr are set, --force-ocr wins.

Additionally the OCR ground-truth files were reorganized from a flat, loosely-tagged layout into a structured per-engine / per-mode hierarchy:

• Renamed tests/data/scanned/ as tests/data/ocr/.
• GT files now live under groundtruth/<engine>/<mode>/ and are named <doc>.<engine>.<mode>.<suffix> (e.g. ocr_test.tesseract.full_page.json), for each of the four artifacts: .json, .md, .doctags.txt, .pages.meta.json.
• Regenerated/added GT across the OCR engines and modes.

Update documentation.

Checklist:

• Documentation has been updated, if necessary.
• Examples have been added, if necessary.
• Tests have been added, if necessary.

[github-actions]

✅ DCO Check Passed

Thanks @nikos-livathinos, all your commits are properly signed off. 🎉

[mergify]

Merge Protections

🔴 1 of 2 protections blocking · waiting on 👀 reviews

	Protection	Waiting on
🔴	Require two reviewer for test updates	👀 reviews
🟢	Enforce conventional commit	—

🔴 Require two reviewer for test updates

Waiting for

• #approved-reviews-by >= 2

This rule is failing.

When test data is updated, we require two reviewers

• #approved-reviews-by >= 2

---

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

[codecov]

Codecov Report

❌ Patch coverage is 76.11940% with 64 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
docling/utils/visualization.py	15.38%	22 Missing ⚠️
docling/models/base_ocr_model.py	89.58%	10 Missing ⚠️
...erimental/pipeline/threaded_layout_vlm_pipeline.py	0.00%	9 Missing ⚠️
...ges/page_preprocessing/page_preprocessing_model.py	10.00%	9 Missing ⚠️
docling/cli/main.py	25.00%	6 Missing ⚠️
...odels/stages/layout/layout_postprocessing_model.py	95.00%	2 Missing ⚠️
docling/datamodel/pipeline_options.py	96.00%	1 Missing ⚠️
docling/models/stages/layout/layout_model.py	66.66%	1 Missing ⚠️
...els/stages/layout/layout_object_detection_model.py	0.00%	1 Missing ⚠️
docling/models/stages/ocr/kserve_v2_ocr_model.py	0.00%	1 Missing ⚠️
... and 2 more

📢 Thoughts on this report? Let us know!