Commit ec939ad
authored
Font weight names heavier than bold (e.g. FranklinGothic-Heavy) were not
flagged as bold because the detection only matched "bold"/"_bd". Add
"heavy" to both font-name checks, matching the fix reported in #131.
Black is a 900 weight, heavier than Bold, and sits alongside Heavy at the top
of the weight scale (... Semibold, Bold, ExtraBold, Heavy, Black, ExtraBlack,
Ultra). The existing "bold" substring already covers Bold, ExtraBold, UltraBold
and Semi/Demibold, and the previous commit added Heavy, which left Black as the
only common weight above bold still reported as regular.
Measured over 400 documents of the GROBID end-to-end corpus (4.15M tokens,
raw font names via -fullFontName):
bold tokens, master 81,153
bold tokens, with "heavy" and "black" 149,861 (+68,708)
Of that gain "black" accounts for roughly 63,000 tokens and "heavy" for 5,700,
so Black is by a wide margin the larger of the two omissions. 67 of the 400
documents are affected; token counts are unchanged in all of them, i.e. the
change is style-only and does not alter segmentation or extracted text.
All 135 distinct font names containing "black" in that corpus are genuine
900-weight faces (Avenir LT Std Black and its oblique). No blackletter or
decorative faces (Blackadder, Blackoak, Fraktur) appear, so the substring does
not produce false positives here.
Deliberately not matched:
- "ext": ambiguous. Extended is a width designation, not a weight, and the
corpus hits are all false positives (tex_cm_maths_extension,
stixmathextensions, mt-extra, and charterbt-roman whose subset prefix happens
to end in "ext").
- "blk", "ultra", "-bd": real but negligible here (15, 8 and 7 tokens). Bare
abbreviations such as "bl" or "x" would match far too much; if they are ever
wanted they should be delimiter-anchored the way "_bd" already is.
* Don't match font style keywords inside the subset tag
A subsetted font's name begins with six letters and a plus sign, and per
PDF 32000-1 9.6.4 "the choice of letters is arbitrary" -- the tag identifies
a subset, not a style. The style checks ran strstr() over the whole name,
tag included, so the tag could spell a keyword:
ABCDEF+Helvetica -> (no style)
BOLDXY+Helvetica -> bold same font, same text
ITALIC+Helvetica -> italics
The style of a word therefore depended on a tag the producer picked
arbitrarily rather than on the font. Re-saving a document with another tool
changes the tag, so the same document could come out bold or not. It also
works against font deduplication (#233): two subsets of one face can get
different tags and so split into different TextStyle entries.
This applies to every keyword, including the "heavy" and "black" added in the
preceding commits, and to the italic checks.
Only a conformant tag is skipped. Anything else before a '+' belongs to the
real name and is kept -- notably the synthetic "cidfont+fN" names used for
unnamed CID fonts, which account for ~14,500 tokens in the GROBID end-to-end
corpus and would be mangled by cutting at the first '+'. 36 names in that
corpus contain more than one '+', so the tag is matched by position rather
than by searching for a separator.
Anchoring the keywords to the end of the name instead was considered and
rejected: it would lose 52% of genuine matches, because the weight is
routinely followed by another modifier (arial-boldmt, avenirltstd-
blackoblique, arial-bolditalicmt). Only the tag region is ambiguous, so only
it is excluded.
Verified over 400 documents of the GROBID end-to-end corpus: output is
byte-identical to master in all 400. No tag in that corpus (8,586 subsetted
fonts) spells a keyword, which is expected -- the odds are about 1 in 152,000
for "bold" and far lower for the others. This is a correctness fix for a rare
case, not a behaviour change.
The same guard is applied to the disabled TextWord constructor so the two
stay in step if that block is ever re-enabled.
1 parent c28fb9e commit ec939ad
1 file changed
Lines changed: 39 additions & 10 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
553 | 553 | | |
554 | 554 | | |
555 | 555 | | |
| 556 | + | |
| 557 | + | |
| 558 | + | |
| 559 | + | |
| 560 | + | |
| 561 | + | |
| 562 | + | |
| 563 | + | |
| 564 | + | |
| 565 | + | |
| 566 | + | |
| 567 | + | |
| 568 | + | |
| 569 | + | |
| 570 | + | |
| 571 | + | |
| 572 | + | |
| 573 | + | |
| 574 | + | |
| 575 | + | |
556 | 576 | | |
| 577 | + | |
557 | 578 | | |
558 | 579 | | |
559 | 580 | | |
560 | 581 | | |
| 582 | + | |
561 | 583 | | |
562 | 584 | | |
563 | 585 | | |
| |||
678 | 700 | | |
679 | 701 | | |
680 | 702 | | |
681 | | - | |
682 | | - | |
| 703 | + | |
| 704 | + | |
| 705 | + | |
| 706 | + | |
| 707 | + | |
683 | 708 | | |
684 | 709 | | |
685 | 710 | | |
686 | 711 | | |
687 | | - | |
688 | | - | |
689 | | - | |
| 712 | + | |
| 713 | + | |
| 714 | + | |
690 | 715 | | |
691 | 716 | | |
692 | 717 | | |
| |||
893 | 918 | | |
894 | 919 | | |
895 | 920 | | |
| 921 | + | |
896 | 922 | | |
897 | 923 | | |
898 | 924 | | |
| |||
1014 | 1040 | | |
1015 | 1041 | | |
1016 | 1042 | | |
1017 | | - | |
1018 | | - | |
| 1043 | + | |
| 1044 | + | |
| 1045 | + | |
| 1046 | + | |
| 1047 | + | |
1019 | 1048 | | |
1020 | | - | |
1021 | | - | |
1022 | | - | |
| 1049 | + | |
| 1050 | + | |
| 1051 | + | |
1023 | 1052 | | |
1024 | 1053 | | |
1025 | 1054 | | |
| |||
0 commit comments