[Performance] RTX 3090 extremely slow (0.014 pages/sec) despite 19GB free VRAM

[NasonZ]

📝 Describe the Output Issue

Performance is extremely slow on RTX 3090 - taking 5-10 minutes to process a single 10-page PDF. GPU has 24GB VRAM but only using 4-5GB with 18-19GB sitting idle. Getting ~0.03 pages/second vs claimed "25 pages/second on H100". Even accounting for hardware differences, this seems far below expected performance.

Main questions:

1. Is 5-10 minutes per 10-page PDF expected performance for RTX 3090?
2. Are there configuration options to better utilize available GPU resources?
3. What realistic performance should I expect from RTX 3090?

📄 Input Document

Test document: 3HANS-1.pdf (218KB, ~10 pages)
Standard academic paper with text, equations, and some figures.

📤 Current Output

Output is correct - markdown is properly generated with accurate text extraction. The issue is purely performance, not quality.

Conversion takes 287-461 seconds (4.8-7.7 minutes) depending on optimization flags:

• Default: 461.4s (7m 41s)
• With RECOGNITION_MODEL_QUANTIZE=true: 299.1s (4m 59s)
• With COMPILE_ALL=true: 287.9s (4m 48s)

Best case: 4.8 minutes for 10 pages = 0.03 pages/second

This is extremely slow for interactive document processing in a RAG pipeline.

✅ Expected Output

Based on documentation claiming "25 pages/second on H100", and considering RTX 3090 has:

• 24GB VRAM
• Ampere architecture with tensor cores
• Plenty of available VRAM (18-19GB unused during processing)

I expected processing to take seconds, not minutes - even accounting for H100 being significantly faster and having more VRAM (80GB), something closer to the reported figures would be reasonable for interactive use.

⚙️ Environment

• Marker version: 1.10.1 (marker-pdf)
• Surya version: 0.17.0 (surya-ocr)
• Python version: 3.12.9
• PyTorch version: 2.8.0+cu128 (CUDA 12.8)
• Transformers version: 4.57.1
• Operating System: WSL2 Ubuntu (Linux 6.6.87.2-microsoft-standard-WSL2)
• GPU: NVIDIA GeForce RTX 3090 (24GB VRAM)
• Driver: 581.29
• CUDA: 13.0

GPU Verification:

>>> import torch
>>> torch.cuda.is_available()
True
>>> torch.cuda.get_device_name(0)
'NVIDIA GeForce RTX 3090'

📟 Command or Code Used

Click to expand

Python API (Single File)

import os
from marker.converters.pdf import PdfConverter
from marker.models import create_model_dict

# Set optimization flags
os.environ["COMPILE_ALL"] = "true"
os.environ["DETECTOR_POSTPROCESSING_CPU_WORKERS"] = "8"
os.environ["OPENBLAS_NUM_THREADS"] = "8"
os.environ["PDFTEXT_CPU_WORKERS"] = "8"
os.environ["OMP_NUM_THREADS"] = "8"

# Load models and convert
model_dict = create_model_dict()
converter = PdfConverter(artifact_dict=model_dict)
result = converter("/path/to/test.pdf")  # Takes 287.9 seconds

Performance Test Results

All tests on same 10-page PDF:

Configuration	Time	Performance	Notes
Default (no optimization)	461.4s (7m 41s)	0.017 pages/s	Baseline
RECOGNITION_MODEL_QUANTIZE=true	299.1s (4m 59s)	0.027 pages/s	35% faster
COMPILE_ALL=true	287.9s (4m 48s)	0.028 pages/s	38% faster

Most recent test (models cached):

Models loaded in: 18.50s
Recognizing Layout: 100%|███████| 10/10 [00:25<00:00,  2.52s/it]
Running OCR Error Detection: 100%|███████| 1/1 [00:01<00:00,  1.22s/it]
Detecting bboxes: 100%|███████| 1/1 [00:21<00:00, 21.69s/it]
Recognizing Text: 100%|███████| 40/40 [06:22<00:00,  9.56s/it]
Recognizing Text: 100%|███████| 11/11 [00:50<00:00,  4.55s/it]
Recognizing tables: 100%|███████| 1/1 [00:06<00:00,  6.05s/it]
Detecting bboxes: 100%|███████| 1/1 [00:17<00:00, 17.62s/it]
Recognizing Text: 100%|███████| 19/19 [00:12<00:00,  1.56it/s]
Conversion time: 547.85s
Total time: 566.34s (9.4 minutes)

GPU Utilization During Processing:

• VRAM Used: 4-5GB
• VRAM Free: 18-19GB (unused)
• GPU Utilization: Low

📎 Additional Context

Use Case

Building a RAG pipeline that needs to process user-uploaded PDFs on-demand for interactive Q&A. Current 5-10 minute processing time per document is a killer.

Batch Mode Testing

Also attempted CLI batch mode but encountered issues:

• marker input --output_dir output --workers 3 with COMPILE_ALL=true: CUDA OOM error after 38 minutes despite 19GB free VRAM
• marker input --output_dir output --workers 1 without compilation: No output produced after 10+ minutes

Questions

1. Is this expected performance? Should RTX 3090 really take 5-10 minutes for an 10-page PDF?

2. Recommended configuration for RTX 3090: What environment variables, flags, or settings should I use to maximize performance on this GPU?

3. Hardware requirements: If RTX 3090 can't achieve reasonable performance, what hardware is recommended for interactive document processing? Do I need H100/A100 to get sub-minute processing times?

4. Model compilation OOM: Why does COMPILE_ALL=true cause out-of-memory errors in batch mode when there's 19GB free VRAM? Works fine in single-file mode.

Context from Documentation

README states: "Marker is fast. Using a batch size of 1 on a H100, marker is around 25 pages/second"

My results: 0.028 pages/second on RTX 3090 = ~900x slower than advertised benchmark

Any guidance on achieving reasonable performance would be greatly appreciated.

System Configuration

• Running in WSL2 on Windows 10
• GPU is directly accessible to WSL2 (verified with nvidia-smi and torch.cuda)
• Native Linux environment (not using Windows paths)
• All models cached locally after first run

[clach04]

@NasonZ is it possible you hit #1050 ? PR posted.