fix: avoid lossy supersample of raster-only (scanned) pages in PDFium-rendering backends#3803
Open
wittjeff wants to merge 2 commits into
Open
fix: avoid lossy supersample of raster-only (scanned) pages in PDFium-rendering backends#3803wittjeff wants to merge 2 commits into
wittjeff wants to merge 2 commits into
Conversation
PyPdfiumPageBackend.get_page_image() renders every page at 1.5x the requested scale and PIL-resizes down. For vector content this sharpens the output, but for image-only (scanned) pages it is a lossy resample round-trip of already-rasterized pixels that can destroy borderline OCR text detections (issue docling-project#3587). - Render pages whose content is raster-only (image objects only, the common scanned-document case) directly at the requested scale. - Add PdfBackendOptions.supersample_factor (default 1.5, preserving current behavior for pages with text/vector content); 1.0 disables supersampling entirely. - Skip the PIL resize when the rendered bitmap already has the target pixel dimensions. Resolves docling-project#3587 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Jeff Witt <152964771+witt3rd@users.noreply.github.com>
Contributor
|
✅ DCO Check Passed Thanks @wittjeff, all your commits are properly signed off. 🎉 |
Contributor
Merge Protections🟢 Merge protection satisfied — ready to merge. Show 1 satisfied protection🟢 Enforce conventional commitMake sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
…t backend Address review findings on the initial change: - Hoist the duplicated get_page_image() implementation from PyPdfiumPageBackend and DoclingParsePageBackend into ManagedPdfiumPageBackend, so the raster-only skip and the supersample_factor option apply to the default DoclingParse backend too instead of being silently ignored. - Trim pdfium's per-side ceil() rounding excess by cropping instead of resampling: real scans commonly have fractional media boxes (e.g. 595.2 x 841.9), where a round()-based size gate would have re-triggered the lossy resize the fix is meant to avoid. - Treat pages with annotations, or with form XObjects nested beyond the inspection depth, as not raster-only (annotations are drawn on render but are not page content objects). - Skip the page-object walk entirely when supersample_factor is 1.0. - Document the new option in docs/usage/advanced_options.md. Verified that scanned-fixture conversion output is unchanged vs the stored groundtruth for EasyOCR and RapidOCR (fuzzy e2e verification), so no reference data regeneration is required. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Jeff Witt <152964771+witt3rd@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
get_page_image()in the PDFium-rendering backends renders every page at 1.5× the requested scale and PIL-resizes down. For vector content that supersampling sharpens the output, but for image-only (scanned) pages it is a lossy resample round-trip of already-rasterized pixels that can destroy borderline OCR text detections — see #3587 for isolated evidence (an entire sentence silently lost end-to-end on a degraded scan; page CER 0.184 vs ~0.05 for direct rasterization).This implements suggestions 1 and 2 from the issue:
page.get_objects()(traversing form XObjects, since scans are often form-wrapped — e.g.tests/data_scanned/ocr_test.pdf) and is conservative: pages with annotations, or with content nested beyond the inspection depth, keep supersampling. Pages with any text/vector content are unaffected.PdfBackendOptions.supersample_factor(default1.5, i.e. no behavior change for non-raster pages) makes the factor configurable;1.0restores direct rendering everywhere.Implementation notes:
get_page_image()inPyPdfiumPageBackendandDoclingParsePageBackendis hoisted into their shared baseManagedPdfiumPageBackend, so the fix and the new option apply consistently to both the defaultDoclingParseDocumentBackendandPyPdfiumDocumentBackend(rather than the option being silently ignored by the default backend).ceil()per side while the requested pixel dimensions useround(); on the direct-render path the rounding excess is now cropped instead of resampled. This matters in practice: real scans commonly have fractional media boxes (e.g. 595.2×841.9 pt), where a naive size check would have re-triggered the lossy resize.Verification
pdfiumrender at the same scale; a vector page keeps supersampling by default;supersample_factor=1.0yields a direct render on vector pages through both backends.get_page_image:ocr_test*.pdf) are raster-only, so their render path changes; I replicatedtest_e2e_ocr_conversionlocally for EasyOCR and RapidOCR (onnxruntime, standard +force_full_page_ocr) and the conversions verify against the stored groundtruth unchanged. Tesseract variants could not run in my environment (no tessdata) — CI should confirm.tests/test_backend_pdfium.py,tests/test_backend_docling_parse.py,tests/test_backend_docling_parse_legacy.py,tests/test_pdf_password.pypass;make validate(ruff, ty, tach) is clean.Issue resolved by this Pull Request:
Resolves #3587
Checklist:
docs/usage/advanced_options.md)🤖 Generated with Claude Code