Skip to content

feat: Make ThreadedDoclingParseDocumentBackend the default#3764

Draft
PeterStaar-IBM wants to merge 8 commits into
mainfrom
dev/add-threaded-renderer-as-default
Draft

feat: Make ThreadedDoclingParseDocumentBackend the default#3764
PeterStaar-IBM wants to merge 8 commits into
mainfrom
dev/add-threaded-renderer-as-default

Conversation

@PeterStaar-IBM

Copy link
Copy Markdown
Member

Checklist:

  • Documentation has been updated, if necessary.
  • Examples have been added, if necessary.
  • Tests have been added, if necessary.

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

DCO Check Passed

Thanks @PeterStaar-IBM, all your commits are properly signed off. 🎉

@mergify

mergify Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🔴 1 of 2 protections blocking · waiting on 👀 reviews

Protection Waiting on
🔴 Require two reviewer for test updates 👀 reviews
🟢 Enforce conventional commit

🔴 Require two reviewer for test updates

Waiting for

  • #approved-reviews-by >= 2
This rule is failing.

When test data is updated, we require two reviewers

  • #approved-reviews-by >= 2

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

@codecov

codecov Bot commented Jul 7, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.90909% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
docling/backend/docling_parse_backend.py 87.50% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
@PeterStaar-IBM PeterStaar-IBM marked this pull request as ready for review July 9, 2026 12:04
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
…oclang xml)

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
@cau-git cau-git changed the title feat: adding the ThreadedDoclingParseDocumentBackend feat: Make ThreadedDoclingParseDocumentBackend the default Jul 14, 2026
@cau-git cau-git marked this pull request as draft July 14, 2026 07:46
@cau-git

cau-git commented Jul 15, 2026

Copy link
Copy Markdown
Member

Testing with docling-parse 7.8.0 on docling 2.112.0 (2026-07-15)

Below the results of an evaluation run on the first 1000 doclaynet test set samples.

Non-threaded docling-parse backend (using pypdfium renderer)

================================================================================
CONVERSION THROUGHPUT
================================================================================
Pages: 1000
Wall time: 842.12s
Rate: 1.19 pages/s

================================================================================
TEXT METRICS
================================================================================
+-----------+------+-----------+----------+-------------+------------+---------------+--------------+------------------+-----------------+-------------+------------+---------------+--------------+
| Dataset   |    n |   f1 mean |   f1 med |   prec mean |   prec med |   recall mean |   recall med |   edit_dist mean |   edit_dist med |   bleu mean |   bleu med |   meteor mean |   meteor med |
+===========+======+===========+==========+=============+============+===============+==============+==================+=================+=============+============+===============+==============+
| dla/test  | 1000 |     0.876 |    0.963 |        0.89 |      0.965 |         0.888 |         0.97 |            0.235 |           0.122 |       0.736 |      0.864 |         0.826 |        0.941 |
| TOTAL     | 1000 |     0.876 |    0.963 |        0.89 |      0.965 |         0.888 |         0.97 |            0.235 |           0.122 |       0.736 |      0.864 |         0.826 |        0.941 |
+-----------+------+-----------+----------+-------------+------------+---------------+--------------+------------------+-----------------+-------------+------------+---------------+--------------+

================================================================================
TABLE METRICS (TEDS)
================================================================================
+-----------+-----+-------------+------------+---------------------+--------------------+
| Dataset   |   n |   teds mean |   teds med |   teds_content mean |   teds_content med |
+===========+=====+=============+============+=====================+====================+
| dla/test  | 508 |       0.716 |      0.894 |               0.591 |               0.72 |
| TOTAL     | 508 |       0.716 |      0.894 |               0.591 |               0.72 |
+-----------+-----+-------------+------------+---------------------+--------------------+

================================================================================
TABLE METRICS (TEDS, MATCHED ONLY)
================================================================================
+-----------+-----+-------------+------------+---------------------+--------------------+
| Dataset   |   n |   teds mean |   teds med |   teds_content mean |   teds_content med |
+===========+=====+=============+============+=====================+====================+
| dla/test  | 428 |        0.85 |      0.926 |               0.701 |              0.763 |
| TOTAL     | 428 |        0.85 |      0.926 |               0.701 |              0.763 |
+-----------+-----+-------------+------------+---------------------+--------------------+

Threaded docling-parse backend (using native renderer)

================================================================================
CONVERSION THROUGHPUT
================================================================================
Pages: 1000
Wall time: 805.47s
Rate: 1.24 pages/s

================================================================================
TEXT METRICS
================================================================================
+-----------+------+-----------+----------+-------------+------------+---------------+--------------+------------------+-----------------+-------------+------------+---------------+--------------+
| Dataset   |    n |   f1 mean |   f1 med |   prec mean |   prec med |   recall mean |   recall med |   edit_dist mean |   edit_dist med |   bleu mean |   bleu med |   meteor mean |   meteor med |
+===========+======+===========+==========+=============+============+===============+==============+==================+=================+=============+============+===============+==============+
| dla/test  | 1000 |     0.857 |    0.953 |        0.86 |      0.954 |         0.886 |         0.97 |            0.271 |           0.154 |       0.701 |      0.838 |         0.809 |        0.921 |
| TOTAL     | 1000 |     0.857 |    0.953 |        0.86 |      0.954 |         0.886 |         0.97 |            0.271 |           0.154 |       0.701 |      0.838 |         0.809 |        0.921 |
+-----------+------+-----------+----------+-------------+------------+---------------+--------------+------------------+-----------------+-------------+------------+---------------+--------------+

================================================================================
TABLE METRICS (TEDS)
================================================================================
+-----------+-----+-------------+------------+---------------------+--------------------+
| Dataset   |   n |   teds mean |   teds med |   teds_content mean |   teds_content med |
+===========+=====+=============+============+=====================+====================+
| dla/test  | 508 |       0.542 |       0.76 |               0.437 |               0.54 |
| TOTAL     | 508 |       0.542 |       0.76 |               0.437 |               0.54 |
+-----------+-----+-------------+------------+---------------------+--------------------+

================================================================================
TABLE METRICS (TEDS, MATCHED ONLY)
================================================================================
+-----------+-----+-------------+------------+---------------------+--------------------+
| Dataset   |   n |   teds mean |   teds med |   teds_content mean |   teds_content med |
+===========+=====+=============+============+=====================+====================+
| dla/test  | 330 |       0.835 |      0.901 |               0.673 |              0.729 |
| TOTAL     | 330 |       0.835 |      0.901 |               0.673 |              0.729 |
+-----------+-----+-------------+------------+---------------------+--------------------+

Conclusions

  • Text metrics with threaded docling-parse look slightly worse but not by much
  • Table metrics with threaded docling-parse look slightly worse for those tables that could be matched
  • Critical: By far less tables could be matched with threaded docling-parse than with classic serial version (330 vs 428 out of 508 toal). This implies that the threaded docling-parse renderer creates output in which the current layout model often misclassifies tables. Needs further investigation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants