Skip to content

feat: add HAR backend and Office macro detection#3707

Open
AyaanB-DotCom wants to merge 8 commits into
docling-project:mainfrom
AyaanB-DotCom:feat/har-backend-and-macro-detection
Open

feat: add HAR backend and Office macro detection#3707
AyaanB-DotCom wants to merge 8 commits into
docling-project:mainfrom
AyaanB-DotCom:feat/har-backend-and-macro-detection

Conversation

@AyaanB-DotCom

Copy link
Copy Markdown

Add support for converting HAR (HTTP Archive) files to DoclingDocument, and
warn when Office files (docx/xlsx/pptx) contain VBA macros before conversion.

Changes

  • docling/backend/har_backend.py: new HarDocumentBackend — converts HAR
    log.entries into headings, status text, and code blocks; caps inline body
    previews at 2000 chars; skips binary MIME types
  • docling/utils/office_utils.py: warn_if_macros() utility — inspects OOXML
    ZIPs for vbaProject.bin, logs a warning, never blocks conversion
  • docling/datamodel/base_models.py: added InputFormat.HAR with extensions
    and MIME type
  • docling/datamodel/document.py: added HAR entry to _mime_from_extension
  • docling/document_converter.py: added HarFormatOption
  • docling/backend/msword_backend.py, msexcel_backend.py,
    mspowerpoint_backend.py: call warn_if_macros() before loading
  • tests/test_backend_har.py: 13 tests covering format registration, path/stream
    loading, content output, binary exclusion, converter integration
  • tests/test_office_macros.py: 7 tests covering warning, silence on clean files,
    BytesIO rewind behaviour

Checklist:

  • Tests have been added
  • Documentation has been updated (not required — no user-facing API change)
  • Examples have been added (not required)

@github-actions

github-actions Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

DCO Check Passed

Thanks @AyaanB-DotCom, all your commits are properly signed off. 🎉

@mergify

mergify Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🔴 1 of 2 protections blocking · waiting on 👀 reviews

Protection Waiting on
🔴 Require two reviewer for test updates 👀 reviews
🟢 Enforce conventional commit

🔴 Require two reviewer for test updates

Waiting for

  • #approved-reviews-by >= 2
This rule is failing.

When test data is updated, we require two reviewers

  • #approved-reviews-by >= 2

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

I, AyaanB-DotCom <a.bhatti3005@gmail.com>, hereby add my Signed-off-by to this commit: b65f18f
I, AyaanB-DotCom <a.bhatti3005@gmail.com>, hereby add my Signed-off-by to this commit: 2fa042a

Signed-off-by: AyaanB-DotCom <a.bhatti3005@gmail.com>
@codecov

codecov Bot commented Jun 26, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.24138% with 9 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
docling/backend/har_backend.py 91.46% 7 Missing ⚠️
docling/utils/office_utils.py 90.47% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

@ceberam

ceberam commented Jun 26, 2026

Copy link
Copy Markdown
Member

Thanks @AyaanB-DotCom for this PR and your first contribution!

I guess you have edited some files on a Windows system that changed all the line endings. As a result, your PR shows a massive diff in some files (e.g., 2796 additions and 2794 deletions in base_models.py), caused by line ending changes, not actual code changes. This makes the PR review on the Git UI very impractical. Could you please normalize the line endings of the files that you edited?

For instance, on Windows:

dos2unix docling/datamodel/base_models.py

or on MacOS:

sed -i 's/\r$//' docling/datamodel/base_models.py

Btw, I have just opened another PR #3709 to prevent this issue.

Another recommendation for your next contribution: please open separate PRs if you are addressing two different and disconnected topics.

@ceberam ceberam added enhancement New feature or request docx issue related to docx backend pptx issue related to pptx backend xlsx issue related to xlsx backend labels Jun 26, 2026

@dolfim-ibm dolfim-ibm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Security Review

Scope: the Office backend diffs (msword_backend.py, msexcel_backend.py, mspowerpoint_backend.py) are pure line-ending (CRLF→LF) churn with no logic changes and are excluded from this review. The analysis covers the two genuinely new files: docling/backend/har_backend.py and docling/utils/office_utils.py.

Overall verdict: needs changes before merge — three medium-severity gaps.

The contributor shows clear security awareness (per-body cap, binary MIME exclusion, stdlib JSON, no URL fetching, BytesIO seek-back), but the following gaps should be addressed:


🟠 MEDIUM — F-1: No file-size guard before reading HAR into memory (har_backend.py · __init__)

The constructor reads the entire document into a Python string with no size check:

raw = Path(self.path_or_stream).read_text("utf-8")   # path branch
raw = self.path_or_stream.getvalue().decode("utf-8")  # stream branch

A crafted (or simply large real-world) HAR can be several GB and will load entirely into RAM. Suggested fix:

_MAX_HAR_BYTES = 100 * 1024 * 1024  # 100 MB

# path branch
path = Path(self.path_or_stream)
if path.stat().st_size > _MAX_HAR_BYTES:
    raise DocumentLoadError("HAR file exceeds size limit")
raw = path.read_text("utf-8")

# stream branch
buf = self.path_or_stream.getvalue()
if len(buf) > _MAX_HAR_BYTES:
    raise DocumentLoadError("HAR stream exceeds size limit")
raw = buf.decode("utf-8")

🟠 MEDIUM — F-2: Unbounded entry iteration (har_backend.py · convert())

_MAX_BODY_CHARS = 2000 correctly caps individual bodies, but the for entry in entries loop has no bound on the number of entries. A crafted HAR with 1 million 1-byte entries still produces 1 million document nodes. Suggested fix:

_MAX_ENTRIES = 10_000

if len(entries) > _MAX_ENTRIES:
    _log.warning("HAR contains %d entries; truncating to %d.", len(entries), _MAX_ENTRIES)
    entries = entries[:_MAX_ENTRIES]

for entry in entries:
    ...

🟠 MEDIUM — F-3: Zip-slip not checked in warn_if_macros (office_utils.py)

msexcel_backend._parse_threaded_comments already validates ZIP entry names before accessing members:

if any(m.startswith("/") or ".." in m for m in zip_file.namelist()):
    _log.warning("Skipping file with unsafe ZIP paths")
    return threaded_comments

warn_if_macros performs the same ZIP inspection but skips this check. The inconsistency should be resolved. Also worth narrowing the catch clause — except (zipfile.BadZipFile, Exception) makes the first term redundant and silences MemoryError; prefer except (zipfile.BadZipFile, OSError, ValueError).

with zipfile.ZipFile(path_or_stream) as zf:
    names = zf.namelist()
    if any(n.startswith("/") or ".." in n for n in names):
        _log.warning("Skipping archive with unsafe ZIP entry paths")
        return
    has_macros = any(_VBA_MARKER in name for name in names)

✅ What's done well

  • _MAX_BODY_CHARS = 2000 — per-body cap on both request and response bodies, covered by tests.
  • _is_text_mime() — binary/base64 blobs (images, fonts, etc.) are correctly excluded.
  • No URL fetchingurl and method fields are written as plain text; SSRF not possible.
  • json.loads() — standard-library parser, no eval, JSONDecodeError correctly re-raised as DocumentLoadError.
  • BytesIO seek-backfinally block in warn_if_macros restores stream position; covered by test_bytesio_rewound_after_check.

I, AyaanB-DotCom <a.bhatti3005@gmail.com>, hereby add my Signed-off-by to this commit: b65f18f
I, AyaanB-DotCom <a.bhatti3005@gmail.com>, hereby add my Signed-off-by to this commit: 2fa042a
I, AyaanB-DotCom <a.bhatti3005@gmail.com>, hereby add my Signed-off-by to this commit: 52a0034
I, Ayaan B <a.bhatti3005@gmail.com>, hereby add my Signed-off-by to this commit: 2a40838

Signed-off-by: AyaanB-DotCom <a.bhatti3005@gmail.com>
@AyaanB-DotCom AyaanB-DotCom force-pushed the feat/har-backend-and-macro-detection branch from 99f543b to 616e16d Compare June 26, 2026 14:56
I, AyaanB-DotCom <a.bhatti3005@gmail.com>, hereby add my Signed-off-by to this commit: b65f18f
I, AyaanB-DotCom <a.bhatti3005@gmail.com>, hereby add my Signed-off-by to this commit: 2fa042a
I, AyaanB-DotCom <a.bhatti3005@gmail.com>, hereby add my Signed-off-by to this commit: 52a0034
I, Ayaan B <a.bhatti3005@gmail.com>, hereby add my Signed-off-by to this commit: 2a40838
I, AyaanB-DotCom <a.bhatti3005@gmail.com>, hereby add my Signed-off-by to this commit: 616e16d

Signed-off-by: AyaanB-DotCom <a.bhatti3005@gmail.com>
@AyaanB-DotCom AyaanB-DotCom requested a review from dolfim-ibm June 26, 2026 23:17
@dolfim-ibm

Copy link
Copy Markdown
Member

@AyaanB-DotCom for moving on the review of this PR, can you please update the branch with the latest changes from main?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docx issue related to docx backend enhancement New feature or request pptx issue related to pptx backend xlsx issue related to xlsx backend

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants