feat(ocr): install the OCR engine and languages on demand instead of bundling them

[samuelsl27]

Description of Changes

Reworked from "bundle Tesseract in the installer" to "install it on demand", which
answers the question that stalled this PR: the size.

What changed. The Windows installer no longer carries a Tesseract runtime.
Instead the wizard asks whether text recognition is wanted and in which languages,
and fetches it while installing, with the progress bar moving. Languages can be
added or removed later from the OCR tool or from Settings, and take effect
immediately.

Why. The bundle was 131.97 MB on disk in every download, for a feature not
everyone uses — and 96.77 MB of that (73%) was a single file, libtesseract-5.dll.
Language models were only 16 MB of it. Packaged for download, the engine is
37.5 MB, and it ships in the installer not at all.

I owe a correction on my earlier answer here: the "+97 MB installer delta" I quoted
compared this project's published MSI against an NSIS package I had built
locally. Measured properly — same machine, same JAR, same config, changing only the
resources line — embedding the runtime costs 42.7 MB in the MSI, not 97.

The installer this branch actually produces is 267.0 MB, against 258.3 MB for
the published one. The 8.7 MB between them is version drift (built against 2.14.2,
published is 2.14.3), not this feature: what this PR adds to the download is nothing
at all.

How components are found. The application has no download URL compiled into it.
It knows one address: a manifest listing, per platform, the engine and every
language model with its size and SHA-256. That indirection is deliberate — whoever
publishes the manifest decides which engine build installations receive, can
withdraw or replace a bad artefact without a new release of Stirling-PDF, and can
move the hosting anywhere. system.ocr.manifestUrl makes it configurable, which is
also what lets an air-gapped or corporate install point at an internal mirror.
Right now it points at a release on my fork; if you want this, point it at
yours and the control is entirely on your side.

Language models are catalogued straight from tessdata_fast at a pinned commit, so
they are never rehosted — only hashed. 125 languages are offered, against the two
the bundle carried.

Safety, since this writes executable code next to the application:
SHA-256 is mandatory and verified before anything is moved into place (an artefact
listed without one is refused); archive entries are resolved against the target and
rejected if they climb out; https or a local file only, because plain http would
let the manifest and its digests be rewritten together; entry-count and
expanded-size ceilings; and the engine is expanded to a sibling directory and only
swapped in once complete, so a failed install leaves the previous state rather than
a runtime that exists but cannot run.

Three bugs found on the way, none of them from this branch

--tessdata-dir was never passed to Tesseract (OCRController), while
AutoRotateController did pass it. So the language dropdown was read from
tessDataPath while the engine resolved tessdata next to its own binary: setting
system.tessdataDir moved the list and left the OCR untouched. That is the
behaviour reported in #6534, where a user added ita.traineddata, saw only Italian
offered, and still had no working Italian OCR.

Fixing it needed a guard, which is the second finding. Measured against Tesseract
5.4.0: pointing --tessdata-dir at a directory of bare .traineddata files — the
layout the documentation tells users to assemble — makes the run print
read_params_file: Can't open pdf, write no output file, and still exit 0. The
pdf argument is the name of a config file read from <tessdata>/configs, not an
output format. Since the code checks rc != 0, that failure would pass unnoticed.
So the flag is only passed when the directory carries configs/pdf, and
HowToUseOCR.md now warns about it.

Forcing OCR on an ordinary 19-page A4 document ran out of memory.
system.maxDPI is documented as "maximum allowed DPI" and defaults to 500.
AutoRotateController reads that as a ceiling and picks its own resolution under it;
OCRController adopted it as the target. At 500 DPI an A4 page is 24 megapixels,
roughly 92 MB, against the 2 GB heap the desktop app starts the backend with. On top
of that PDFBox renders RGB by default while Tesseract converts to greyscale before it
does anything else, so three quarters of that allocation was work being undone.

It now renders at 300 DPI — what Tesseract's own documentation asks for — clamped by
maxDPI so lowering the setting still works, and with ImageType.GRAY: 8.7 MB per
page instead of 92, same recognised text. The DPI code is byte-identical to before
this branch and -Xmx2g has not changed, so this is not something the OCR work
introduced. Happy to lift it into its own PR if you would rather review it apart.

Notes for reviewers

• The endpoints live under /api/v1/ui-data, next to the existing tessdata ones,
  rather than /api/v1/misc. /api/v1/misc/ is in the tool-model generator's
  allow list, so a POST there would be published as a pipeline step, which none of
  this is.
• They are deliberately not hasRole('ADMIN'): the desktop starts the backend with
  security.enableLogin=false, so an admin-gated endpoint is unreachable exactly
  where this feature is needed — which is why the existing tessdata downloader
  cannot be used there.
• windows/wix/main.wxs is forked from tauri-bundler at @tauri-apps/cli v2.10.1
  because Tauri offers no way to add a dialog without replacing the template. The
  provenance is recorded at the top of the file and the change is confined to one
  block. This is the real maintenance cost of the wizard page and I am not hiding
  it; happy to drop that page and keep only the in-app path if you would rather.
• The installer's custom action is a DLL rather than another helper exe because a
  type EXE action runs out of process and never receives the MSIHANDLE, so it
  cannot drive the progress bar.
• Return="ignore" on that action is deliberate: a proxy wanting credentials or a
  laptop losing wifi mid-wizard should not roll back an otherwise good install. It
  logs, leaves a note the app reads on first launch, and lets the install finish.
• Unattended deployment works with no UI:
  msiexec /i Stirling-PDF.msi /qn STIRLING_OCR=1 STIRLING_OCR_LANGS=spa,eng
• I have not run scripts/counter_translation_v3.py: sync_files_v2.yml
  regenerates the README table on main, and running it by hand would add a diff
  across every language to this PR. Say the word if you would rather I did.

Closes #7257

---

Checklist

General

• I have read the Contribution Guidelines
• I have read the Stirling-PDF Developer Guide (if applicable)
• I have read the How to add new languages to Stirling-PDF (if applicable)
• I have performed a self-review of my own code
• My changes generate no new warnings

Documentation

• I have updated relevant docs on Stirling-PDF's doc repo (if functionality has heavily changed)
• I have read the section Add New Translation Tags (for new translation tags only)

Translations (if applicable)

• I ran scripts/counter_translation.py

UI Changes (if applicable)

• Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR)

Testing (if applicable)

• I have run task check to verify linters, typechecks, and tests pass
• I have tested my changes locally. Refer to the Testing Guide for more details.

What I actually ran, and what I did not

Precisely, since the box above is a single tick over several commands: task
itself is not installed on this machine, so I ran what it delegates to. Backend
spotlessCheck clean; the Java suites pass — 3,598 in app/core, 2,612 in
app/proprietary, and app/common clean apart from four ArchitectureTest
failures that are this machine, not the branch (the repository is on an SMB share
here and ArchUnit's importer reads zero classes over UNC; the same importer
reads 395 from a local copy, and the failure message is "failed to check any
classes", not a violation). Frontend: tsc --noEmit, oxlint, stylelint,
theme-lint and prettier --check all clean, and the editor suite is running.
Repo-wide: whitespace, codespell and the locale TOML sort check pass. The
Python engine's checks I did not run — this branch does not touch engine/.

Running it caught three things worth naming, since the point of the gate is that
it catches what re-reading the diff does not:

• Two tests depended on what the host had installed. They asserted the
  Tesseract path was the bare command name, which stops being true the moment the
  MSI actually installs OCR machine-wide — green in CI, red for anyone using the
  feature. The assertion moved to where it belongs.
• Translations had been added to es-ES as well as en-US. AGENTS.md is
  explicit that only en-US is hand-edited. Reverted.
• The locale file was left unsorted for sort_locale_toml.py. Sorted.

One gap remains, and I would rather name it than have a reviewer find it:
no screenshots yet for the Settings section, the tool dialog or the installer
page. They are owed.

The wizard page is no longer unproven. An earlier revision of this section
said the installer page had never been through msiexec. It has now: the existing
release was uninstalled and the MSI built from this branch installed in its place.
The OCR page appears where intended; the engine downloads during installation with
its SHA-256 verified before anything is moved into place; configs/ arrives with
it; adding a language afterwards from Settings works and the application finds the
engine the installer left rather than downloading a second copy; the ACL grants
Modify to Users on tessdata only; and uninstalling removes the machine-wide copy.

[Frooodle]

What kind of size is the tesseract.exe bundle within the installer? What does the installer size delta?

[samuelsl27]

What kind of size is the tesseract.exe bundle within the installer? What does the installer size delta?

Bundle: 132 MB. Installer: 258.4 MB → 355.4 MB, so a delta of ~97 MB.

Measured on the NSIS installer built from this branch, against the current release installer.

The bundle is trimmed rather than the whole Tesseract distribution. Unpacking the UB Mannheim installer gives 238 MB; dropping the training tools (lstmtraining, text2image and friends) and the Pango/Cairo/ICU stack they pull in takes it to 132 MB. Breakdown:
libtesseract-5.dll | 96.8 MB
25 other DLLs | ~19 MB
tesseract.exe | 1.5 MB
eng + spa + osd models | ~16 MB
Total | 132 MB

The DLL list was derived by elimination — removing each one and re-running OCR — and verified against all three ways Stirling-PDF drives Tesseract: text output, pdf output (which needs tessdata/configs/pdf) and --psm 0 orientation detection (which needs osd.traineddata).

Also fixed the -Languages bug the automated review caught (cb05995): the already-built check ignored the requested languages, so adding one to an existing bundle exited early and downloaded nothing. Good catch — it failed silently.

On the other Aikido finding, invoking tesseract.exe from $OutputDir: that parameter is supplied by whoever runs the build, not by any external input, and the path defaults to a location inside the repo.

[balazs-szucs]

Isn't this a better solution: https://docs.spring.io/spring-boot/api/java/org/springframework/boot/system/ApplicationHome.html

It feels like you are essentially recreating what ApplicationHome provides using less-then sensible hardcoded heuristics. https://docs.spring.io/spring-boot/api/java/org/springframework/boot/system/ApplicationHome.html#getDir() is direct replacement to a good chunk of this PR.

[samuelsl27]

Isn't this a better solution: https://docs.spring.io/spring-boot/api/java/org/springframework/boot/system/ApplicationHome.html

It feels like you are essentially recreating what ApplicationHome provides using less-then sensible hardcoded heuristics. https://docs.spring.io/spring-boot/api/java/org/springframework/boot/system/ApplicationHome.html#getDir() is direct replacement to a good chunk of this PR.

You're right, this is a better solution, I change it

[aikido-pr-checks]

HTTP request might enable SSRF attack - medium severity
If an attacker can control the URL input leading into this http request, the attack might be able to perform an SSRF attack. This kind of attack is even more dangerous is the application returns the result of the URL fetch to the user. It can serve as an initial access point for an attacker for stealing credentials in the cloud.

Show fix

Remediation: If possible, only allow requests to verified domains. If not, consult the article linked above to learn about other mitigating techniques such as disabling redirects, blocking private IPs and making sure private services have internal authentication. If you return data coming from the request to the user, validate the data before returning it to make sure you don't return random data.

_{Reply @AikidoSec ignore: [REASON] to ignore this issue.
More info}

[samuelsl27]

Good catch, and it was right about more than it could see. Fixed in f49578b4.

Two real holes, not one:

• The guard was at the download call sites, so loadManifest went straight past it — the catalogue URL itself was never checked.
• The client followed redirects automatically, so a hop was taken after the check. A public catalogue could answer 302 into the internal network and the guard would never see the real destination. Your remediation text names disabling redirects for exactly this reason.

Both are closed by moving the check into open(), the one place this class touches the network, and following redirects by hand with the guard re-applied to every hop. Refusing redirects outright was not an option — a GitHub release asset takes one — so it is a bounded manual loop instead.

The check reuses the address ranges already vetted in GeneralUtils rather than a second implementation. It is not a flat ban on private addresses, because that would break the case this feature exists for: an air-gapped or corporate install points system.ocr.manifestUrl at an internal mirror whose artefacts are on the same internal network. So the rule follows the trust — an internal catalogue may name internal artefacts, a public one may not name anything that resolves inside.

Worth recording that writing this introduced a bug the tests then caught: asking the existing helper whether the catalogue was internal conflates "resolves inside" with "cannot be resolved", so a DNS failure on the catalogue host read as "internal mirror" and disabled the guard entirely. That is now a separate check that only says internal when the host really does resolve inside.

Everything fetched is still verified against the SHA-256 the catalogue declares before anything is kept, and only https or a local file is accepted.

[aikido-pr-checks]

HTTP request might enable SSRF attack - medium severity
If an attacker can control the URL input leading into this http request, the attack might be able to perform an SSRF attack. This kind of attack is even more dangerous is the application returns the result of the URL fetch to the user. It can serve as an initial access point for an attacker for stealing credentials in the cloud.

Show fix

Remediation: If possible, only allow requests to verified domains. If not, consult the article linked above to learn about other mitigating techniques such as disabling redirects, blocking private IPs and making sure private services have internal authentication. If you return data coming from the request to the user, validate the data before returning it to make sure you don't return random data.

_{Reply @AikidoSec ignore: [REASON] to ignore this issue.
More info}

[samuelsl27]

@AikidoSec ignore: the two concrete holes this rule helped find are fixed in f49578b4; what is left is the pattern itself, which is inherent to fetching a component described by a configurable manifest.

For a reviewer deciding whether that is reasonable, the mitigations now in place:

• open() is the only place this class touches the network, and the guard runs at the top of every request, including the catalogue fetch.
• Redirects are followed by hand with the guard re-applied to each hop, so a 302 cannot smuggle a destination past the check. Bounded to five hops.
• Only https or a local file is accepted; plain http is refused, because it would let the manifest and the digests inside it be rewritten in the same breath.
• Destinations are checked against the address ranges already vetted in GeneralUtils, not a second implementation of the same idea.
• Every artefact is verified against the SHA-256 the catalogue declares before anything is kept, and a mismatch installs nothing.
• Response bodies are written to a file and never returned to the caller, so this cannot be used as a read primitive.

The one thing deliberately allowed is an operator pointing system.ocr.manifestUrl at an internal mirror and that mirror naming internal artefacts — that is the air-gapped install case, and anyone who can edit settings.yml already controls the server. A public catalogue may not name anything that resolves inside.

Happy to be told this reasoning is wrong.

[aikido-pr-checks]

✅ Based on your feedback, we ignored this issue because of the following reason:

the two concrete holes this rule helped find are fixed in f49578b4; what is left is the pattern itself, which is inherent to fetching a component described by a configurable manifest.

For a reviewer deciding whether that is reasonable, the mitigations now in place:

• open() is the only place this class touches the network, and the guard runs at the top of every request, including the catalogue fetch.
• Redirects are followed by hand with the guard re-applied to each hop, so a 302 cannot smuggle a destination past the check. Bounded to five hops.
• Only https or a local file is accepted; plain http is refused, because it would let the manifest and the digests inside it be rewritten in the same breath.
• Destinations are checked against the address ranges already vetted in GeneralUtils, not a second implementation of the same idea.
• Every artefact is verified against the SHA-256 the catalogue declares before anything is kept, and a mismatch installs nothing.
• Response bodies are written to a file and never returned to the caller, so this cannot be used as a read primitive.

The one thing deliberately allowed is an operator pointing system.ocr.manifestUrl at an internal mirror and that mirror naming internal artefacts — that is the air-gapped install case, and anyone who can edit settings.yml already controls the server. A public catalogue may not name anything that resolves inside.

Happy to be told this reasoning is wrong.

[samuelsl27]

@Frooodle — updating this with the final numbers, now that the installer has been built and installed for real on a clean machine. Two figures I gave earlier were wrong, and both corrections make this smaller rather than larger.

The "+97 MB" was wrong. It compared this project's published MSI against an NSIS package I had built locally — different formats, so the delta meant nothing. Sorry for the noise.

The absolute figures I posted a few hours ago were also wrong, and that one was my build rather than anything in the project: I had built the JAR without DISABLE_ADDITIONAL_FEATURES=true, which inflated every total by about 82 MB. The delta survived it, since both sides carried the same mistake, but the totals were not comparable to anything and should not have been posted.

What stands, all measured on one machine:

Embedding the runtime, measured like for like	42.7 MB
What this PR adds to the installer	0 MB
This branch's MSI · the published MSI	267.0 MB · 258.3 MB

The 8.7 MB between the two MSIs is version drift — this branch is built against 2.14.2 and the published one is 2.14.3 — not this feature. The engine is fetched only when someone asks for it, at 37.5 MB.

Where the weight actually sits, in case it is useful beyond this PR: libtesseract-5.dll is 96.77 MB of the 131.97 MB bundle, i.e. 73%. The language models are only 16 MB. So "OCR is heavy" is really "one DLL from the UB Mannheim build is heavy" — a minimal rebuild is the obvious lever if anyone wants to pursue it, though --psm 0 (used by auto-rotate) may depend on the legacy engine, which would cap the saving.

The wizard page is no longer unproven

I flagged in the PR body that the installer page had never been through msiexec and that only a real install could confirm it. That is now closed. Existing 2.14.x uninstalled, MSI from this branch installed:

• The OCR page appears in the wizard where intended, with the language list, and unattended install still works with no UI.
• The engine downloads during installation with the progress bar moving, and its SHA-256 is verified before anything is moved into place.
• configs/ arrives with the engine, so --tessdata-dir is safe to pass — that is the failure that prints read_params_file: Can't open pdf, writes nothing, and still exits 0.
• Adding a language afterwards from Settings works and takes effect without a restart, and the application finds an engine the installer put there instead of downloading a second copy beside it.
• The ACL grants Modify to Users on tessdata only, not on the engine directory.
• Uninstall removes the machine-wide copy.

An out-of-memory bug that predates this branch

Worth reporting separately from the feature, because it affects everyone and is not a regression from here. Forcing OCR on an ordinary 19-page A4 document runs out of memory.

system.maxDPI is documented as "maximum allowed DPI" and defaults to 500. AutoRotateController reads that as a ceiling and picks its own resolution under it; OCRController adopted it as the target. At 500 DPI an A4 page is 24 megapixels, roughly 92 MB, against the 2 GB heap the desktop app starts the backend with. On top of that PDFBox renders RGB by default while Tesseract converts to greyscale before it does anything else, so three quarters of that allocation was work being undone immediately.

It now renders at 300 DPI — what Tesseract's own documentation asks for — clamped by maxDPI so lowering the setting still works, and with ImageType.GRAY: 8.7 MB per page instead of 92, same recognised text. The DPI code is byte-identical to before this branch and -Xmx2g has not changed, so this is not something the OCR work introduced; it is just where I happened to be standing when it fell over. Happy to lift it into its own PR if you would rather review it apart from the rest.

[github-actions]

🌐 TOML Translation Verification Summary

🔄 Reference Branch: pr-branch

📃 File Check: en-US/translation.toml

1. Test Status: ✅ Passed
2. Test Status: ✅ Passed
3. Test Status: ✅ Passed

---

✅ Overall Check Status: Success

Thanks @samuelsl27 for your help in keeping the translations up to date.