fix(layout): recover heading text wrongly absorbed into PICTURE regions

[DanielNg0729]

Recover heading text that gets swallowed by PICTURE regions

The problem

I picked this up from #3699: a heading goes missing from the Markdown output, and it stays missing even with ContentLayer.FURNITURE turned on. That's what makes it a different animal from #3693. In #3693 the heading is only put in the wrong layer; it's sitting in furniture, so asking for furniture brings it back. Here it's gone from every layer, which means something drops it well before the export stage gets a say.

Why it happens

A solid filled box is a strong "this is a picture" signal, so the layout model draws a low-confidence PICTURE over that header region, on page 2 it lands at about 0.63, fighting with overlapping section_header / page_header / title boxes over the exact same text.

From there the postprocessor does what it's designed to do. Any regular cluster that sits ≥ 80% inside a special cluster gets adopted as a child of it (_process_special_clusters), and postprocess() then strips those children out of the regular set — and SPECIAL_TYPES includes PICTURE. So the header's text cells become children of the picture and vanish from the regular clusters. The picture serializes as a figure (<!-- image -->) and never emits its child text, so the heading reaches neither BODY nor FURNITURE.

This is the right behaviour for a real figure: we don't want labels from inside a chart leaking into the body. The trouble is that a text box the model mistook for a figure looks identical to a genuine one at this stage. I checked the whole chain by running the actual layout model on the page and dumping the clusters: the cluster that survives at the top is a picture, with the article title sitting underneath it as absorbed children.

What I changed

A small, opt-in rescue inside LayoutPostprocessor. When a PICTURE is really just a box drawn around text, I drop the picture and leave its text clusters in the regular set, so they come out normally with their own labels — the title goes to BODY, the page number to FURNITURE.

I only treat a PICTURE as "text-only" (_is_text_only_picture) when two things are both true: everything inside it is text-like (TEXT, SECTION_HEADER, PAGE_HEADER, PAGE_FOOTER, TITLE, LIST_ITEM, CAPTION, FOOTNOTE — I left FORMULA/CODE/CHECKBOX out on purpose, so a figure with a formula in it stays safe), and the text cells cover at least 25% of the picture's area. A genuine photo, chart or diagram has little or no text-cell coverage, so it never qualifies. I picked the 25% line by measuring the reporter's PDF — the spurious header pictures come in between 0.31 and 0.44, well clear of a figure that just has a caption.

The change touches three files: the new recover_text_in_pictures flag on BaseLayoutOptions, the _is_text_only_picture check plus the drop in _process_special_clusters, and a new test file.

How to use it

It's off by default, so nobody's output changes unless they ask for it:

opts = PdfPipelineOptions()
opts.layout_options.recover_text_in_pictures = True

On the reporter's PDF that's the difference between the heading being absent with a stray <!-- image --> in its place (default), and the heading actually showing up — title in the body, number in furniture (flag on).

Tests

tests/test_layout_recover_text_in_pictures.py exercises LayoutPostprocessor directly with hand-built clusters, so it's deterministic and needs neither the layout model nor a PDF fixture. It checks the decision on its own (a text box gets rescued; a real figure with a small caption is left alone; a figure containing a formula is left alone) and the full _process_special_clusters path both ways. Flag on, the picture's gone and the text survives; flag off, the picture keeps the text exactly as before.

Honest limitations

It's a heuristic, and I want to be upfront about where it's soft. If you turn it on, a figure that genuinely is mostly text — a screenshot of a page, a busy infographic — could get reclassified from a picture into text. The all-text-like guard plus the 25% floor keep that corner small, but not zero, which is the main reason this is opt-in rather than default.

There's also a reading-order rough edge: the recovered running header currently lands at the bottom of the page rather than the top where you actually see it, because the rescued cells come back as orphan clusters ordered by native cell index. It's present and correct, just not where you'd want it visually. And because this header repeats on every page, turning the flag on surfaces it on every page — the cross-page de-duplication of headers/footers is the separate problem in #3693, and I deliberately kept it out of here. Finally, the 25% threshold is calibrated on the one PDF in the report (0.31–0.44 on its header pages), so it may want revisiting as more samples turn up. This is a post-processing rescue, not a change to the model, so it never fires unless the picture genuinely contains the text (≥ 80% containment).

A couple of things I'd like your take on

• Is 25% plus that label set the right dial, and would you rather the threshold be a real option than a constant?
• Should the reading-order placement and per-page repetition be sorted here, or left to the header/footer work in Obvious non-repeating header removed #3693?

Resolves #3699.

---

Checklist:

• Documentation has been updated, if necessary.
• Examples have been added, if necessary.
• Tests have been added, if necessary.

[github-actions]

✅ DCO Check Passed

Thanks @DanielNg0729, all your commits are properly signed off. 🎉

[mergify]

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

[codecov]

Codecov Report

❌ Patch coverage is 93.33333% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
docling/utils/layout_postprocessor.py	92.85%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!