Skip to content

fix: emit glyph marker for glyph-index names instead of leaking them as text#302

Open
wittjeff wants to merge 1 commit into
docling-project:mainfrom
wittjeff:fix/gid-glyph-name-fallback
Open

fix: emit glyph marker for glyph-index names instead of leaking them as text#302
wittjeff wants to merge 1 commit into
docling-project:mainfrom
wittjeff:fix/gid-glyph-name-fallback

Conversation

@wittjeff

Copy link
Copy Markdown
Contributor

Addresses the glyph-name half of #238.

Problem

Subset generators (FontForge, fontTools, mPDF, ...) name glyphs by bare index: /gid00043, /g43, /glyph43. Such a name identifies the glyph inside the embedded font program but carries no reading text. When /Encoding/Differences assigns these names and no /ToUnicode CMap covers the code, init_differences()'s last-resort fallback kept the raw name as reading text:

/Differences [ 65 /gid00043 /gid00049 /gid00041 ]  →  "gid00043gid00049gid00041"

Because that is plausible-looking text rather than a GLYPH<...> marker, downstream consumers cannot detect the corruption — docling's rate_text_quality() gate never fires and no OCR fallback triggers. This is the same fabricated-text class as #299, reached through the glyph-name path instead of the encoding path.

(Note: the with-ToUnicode variant of #238 is already fixed on maininit_differences() consults the CMap first per PDF 32000-1 §9.10.2. Released 5.11.0 still emits /gidNNNNN even when a valid ToUnicode exists. And the file attached to #238 does not reproduce for me on 5.11.0 or main — details in the issue thread.)

Fix

The last-resort fallback (resolve_unknown_name) now detects pure glyph-index names — (gid|glyph|g|cid|index)\d+, case-insensitive, full-match — and emits a GLYPH<name:gid00043> marker instead of the name. Under the default keep_glyphs=false the marker is stripped to a space, so no fabricated text reaches consumers; with keep_glyphs=true the glyph identity is preserved for diagnostics.

Deliberately unchanged:

  • a valid /ToUnicode CMap stays authoritative ahead of the guard (method 1 of §9.10.2);
  • meaningful unknown names (custom ligatures like /Th, /ft) keep the existing name-as-text fallback;
  • glyph-name based methods (AGL tables, uniXXXX, ligature join/decompose, /C<digits>) all run before the guard.

Recovering the actual text for these fonts would require reverse-looking-up the embedded font program's cmap table (gid → Unicode) — worth discussing as a follow-up; this PR makes the corruption detectable so OCR fallback can kick in meanwhile.

Tests

tests/test_gid_glyph_names.py builds the PDFs in memory (same pattern as #299's tests): gid-named Differences with and without ToUnicode, name variants (g43, glyph49, index41), and a regression guard for ligature-style names. Full suite passes (88 tests).

…as text

Subset generators (FontForge, fontTools, mPDF, ...) name glyphs by bare
index: /gid00043, /g43, /glyph43. When /Encoding/Differences carries such
names and no /ToUnicode CMap covers the code, init_differences() kept the
raw name as reading text, so extraction produces plausible-looking
garbage ('gid00043gid00049...') that downstream quality gates cannot
detect (docling-project#238).

Detect pure glyph-index names in the last-resort fallback and emit a
GLYPH<name:...> marker instead, keeping the unresolved glyph detectable
(and stripped to a space under the default keep_glyphs=false, so no
fabricated text reaches consumers). A valid /ToUnicode CMap stays
authoritative ahead of the guard, and meaningful unknown names (custom
ligatures like /Th) keep the existing name-as-text fallback.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Jeff Witt <152964771+witt3rd@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

DCO Check Passed

Thanks @wittjeff, all your commits are properly signed off. 🎉

@mergify

mergify Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants