Docling with the transformers library

[Justinius]

I have an environment where I can load models with VLLM, but the transformers library isn't really available as an option as a dependency - its a long story.

Is it possible to use granite-docling through VLLM to convert pdfs to md - I know my pds have a mix of some with text layers and some scanned, so I'm looking at what my options are with docling.

[dosubot[bot]]

Hi @Justinius! I'm Dosu and I’m helping the docling team.

You can't use granite-docling through VLLM for PDF-to-Markdown conversion without the transformers library. Even when you select VLLM as the inference backend, docling's VllmVlmModel still imports and uses transformers components (like AutoProcessor) for prompt templating and image processing, and the vLLM engine is initialized with model_impl="transformers"—so transformers is a required dependency for this workflow (source).

However, for scanned PDFs (images), docling's OCR pipeline is completely independent of transformers. You can use Tesseract, EasyOCR, RapidOCR, or macOS OCR for text extraction from scanned PDFs by configuring the OCR backend in pipeline options. This pathway does not require transformers at all (source). For PDFs with text layers, docling can extract text directly without transformers as well.

So, if you only need OCR-based extraction (for scanned/image PDFs), you can use docling without transformers. But if you want to use granite-docling or any VLM-based PDF-to-Markdown conversion through VLLM, transformers is still required as a dependency.

To reply, just mention @dosu.

---

Share context across your team and agents. Try Dosu.

[dolfim-ibm]

We are soon going to have a docling-slim package (see #2535) which will have minimum dependencies. I'm not yet sure about transformers, but torch should definitely be optional ehre.