Clean up entrypoints and flags; budget batch VLM concurrency

Batch parallelization:
- The parent now reads the inference server's capacity after spawn and
  budgets aggregate client concurrency across workers (~1.5x capacity)
  instead of every worker independently fanning out to full capacity
- --disable_ocr batch runs skip the server entirely and size the pool
  by CPU cores
- maxtasksperchild 50 -> 200 (workers only hold small CPU models now)
- --num_chunks/--chunk_idx re-documented as multi-node file-list
  sharding (one marker invocation per node, each with its own server)

Cruft removal:
- Delete marker_chunk_convert + chunk_convert.{py,sh}: the per-GPU
  wrapper was broken by design (all chunks attach to the first chunk's
  server) and is superseded by VLLM_GPUS + multi-node sharding
- Delete dead config fields: LineMergeProcessor.min_merge_ydist,
  LLMTableProcessor.table_image_expansion_ratio,
  LLMTableMergeProcessor.table_start_threshold, and the unreachable
  large-page reading-order branch (subsumed by
  use_pdftext_reading_order defaulting on)

Fixes:
- ConfigParser no longer drops explicit falsy CLI values (False/0);
  only None (unset) is skipped
- marker_extract: add missing streamlit-ace dependency
- Server API: add mode param (fast/balanced) to /marker endpoints

Claude-Session: https://claude.ai/code/session_01SxGJdyccsAdKbjNAth35MA

Author: VikParuchuri

README.md

@@ -105,7 +105,7 @@ First, some configuration:
 - Marker runs layout, OCR, and table recognition through a single surya VLM, served by a local inference server (used for OCR in both modes, and for layout in balanced mode).  The server is spawned automatically on first use - vLLM (docker) on NVIDIA GPUs, llama.cpp elsewhere.  You can also point marker at an already-running server with `SURYA_INFERENCE_URL=http://host:port/v1`.
 - Useful server settings (all surya env vars): `SURYA_INFERENCE_BACKEND` (`vllm` or `llamacpp`), `SURYA_INFERENCE_PARALLEL` (concurrent requests — by default this auto-scales to the server's capacity: the GPU's `max_num_seqs` under vllm, a conservative slot count under llama.cpp; set an int only to override), `SURYA_INFERENCE_KEEP_ALIVE` (keep the server running between invocations), `VLLM_GPUS` (GPU indices for the server).
 - Some PDFs, even digital ones, have bad text in them.  Set `--force_ocr` to force OCR on all pages, or the `strip_existing_ocr` to keep all digital text, and strip out any existing OCR text.
-- If you care about inline math, set `force_ocr` to convert inline math to LaTeX.
+- Inline math is converted to LaTeX automatically in balanced mode (`ocr_inline_math`); in fast mode, set `--force_ocr` or `--ocr_inline_math` to get the same.
 
 ## Interactive App
 
@@ -130,7 +130,7 @@ Options:
 - `--output_dir PATH`: Directory where output files will be saved. Defaults to the value specified in settings.OUTPUT_DIR.
 - `--paginate_output`: Paginates the output, using `\n\n{PAGE_NUMBER}` followed by `-` * 48, then `\n\n`
 - `--use_llm`: Uses an LLM to improve accuracy.  You will need to configure the LLM backend - see [below](#llm-services).
-- `--force_ocr`: Force OCR processing on the entire document, even for pages that might contain extractable text.  This will also format inline math properly.
+- `--force_ocr`: Force OCR processing on the entire document, even for pages that might contain extractable text.
 - `--block_correction_prompt`: if LLM mode is active, an optional prompt that will be used to correct the output of marker.  This is useful for custom formatting or logic that you want to apply to the output.
 - `--strip_existing_ocr`: Remove all existing OCR text in the document and re-OCR with surya.
 - `--redo_inline_math`: If you want the absolute highest quality inline math conversion, use this along with `--use_llm`.
@@ -152,11 +152,16 @@ marker /path/to/input/folder
 ```
 
 - `marker` supports all the same options from `marker_single` above.
-- `--workers` is the number of conversion workers to run simultaneously.  This is automatically set by default, but you can increase it to increase throughput, at the cost of more CPU usage.  All workers share a single inference server, which the parent process spawns - GPU parallelism is handled server-side (`SURYA_INFERENCE_PARALLEL`).
+- `--workers` is the number of conversion workers to run simultaneously.  This is automatically set by default, but you can increase it to increase throughput, at the cost of more CPU usage.  All workers share a single inference server, which the parent process spawns.
+- The parent budgets total VLM concurrency automatically: it reads the server's capacity and splits it across workers (aggregate in-flight ≈ 1.5× capacity), so adding workers never over-queues the server.  Set `SURYA_INFERENCE_PARALLEL` yourself only to override.
+- With `--disable_ocr` no inference server is started at all, and the pool is sized purely by CPU cores.
 
-## Convert multiple files on multiple GPUs
+### Batch sizing cheat sheet (e.g. 1000 docs)
 
-For multi-GPU machines, run a single `marker` command and let the inference server span GPUs by setting `VLLM_GPUS` (e.g. `VLLM_GPUS=0,1,2,3`).  The old `marker_chunk_convert` script assumed one model instance per GPU - with the shared inference server, chunks on the same machine will attach to the first chunk's server.  To run truly separate servers, start one per GPU manually and pass a distinct `SURYA_INFERENCE_URL` per chunk.
+- **One GPU machine**: `marker /folder --output_dir out` — defaults handle it: one vllm server, a CPU-sized worker pool, concurrency budgeted to the GPU.  Add `--mode fast` if you want cheaper/faster conversion for mostly-digital corpora.
+- **Multi-GPU machine**: same single command, with the server spanning GPUs: `VLLM_GPUS=0,1,2,3 marker /folder ...`
+- **Multiple machines**: shard the file list — run one `marker` per node with `--num_chunks <nodes> --chunk_idx <this node>`.  Each node spawns its own server.
+- **CPU-only / no VLM**: `marker /folder --disable_ocr` (pure text-layer extraction; equations and scanned pages are skipped).
 
 ## Use from python
 

marker/builders/line.py

@@ -76,24 +76,6 @@ class LineBuilder(BaseBuilder):
         "multi-column pages (olmocr-bench order tests: 75% vs 56%). The learned",
         "head is still used on OCR pages, which have no pdftext positions.",
     ] = True
-    large_page_pdftext_order: Annotated[
-        bool,
-        "On very large clean-digital (pdftext) pages - newspapers, broadsheets -",
-        "use the PDF's character reading order instead of the learned order head.",
-        "The order model is trained at a fixed small resolution with a box-count",
-        "cap, so it degrades on big multi-column pages where pdftext's column-aware",
-        "char stream stays reliable.",
-    ] = True
-    large_page_min_long_side: Annotated[
-        int,
-        "Long side (px at lowres dpi) above which a pdftext page counts as large",
-        "for large_page_pdftext_order.",
-    ] = 2000
-    large_page_min_short_side: Annotated[
-        int,
-        "Short side (px at lowres dpi) above which a pdftext page counts as large",
-        "for large_page_pdftext_order.",
-    ] = 1000
     mode: Annotated[
         str,
         "Conversion mode. 'balanced' promotes any page with flagged blocks to",
@@ -155,13 +137,6 @@ def __call__(self, document: Document, provider: PdfProvider):
         if not self.disable_ocr:
             self.flag_bad_blocks(document)
 
-    def _is_large_page(self, page: PageGroup) -> bool:
-        w, h = page.get_image(highres=False).size
-        return (
-            max(w, h) >= self.large_page_min_long_side
-            and min(w, h) >= self.large_page_min_short_side
-        )
-
     def order_blocks_by_reading_order(self, document: Document):
         """Order layout blocks on pdftext pages by the PDF's character reading
         order, instead of the layout model's position.
@@ -175,20 +150,15 @@ def order_blocks_by_reading_order(self, document: Document):
         blocks by ``span.minimum_position`` (pdftext char start index =
         column-aware reading order) instead.
 
-        Applies to a pdftext page when ``use_pdftext_reading_order`` forces it
-        for all pages, or ``large_page_pdftext_order`` and the page is large.
         Text-less blocks (figures, empty boxes) keep their placement relative to
         the text block they followed. OCR'd ("surya") pages are left untouched
         (no pdftext positions - they rely on the learned order head).
         """
+        if not self.use_pdftext_reading_order:
+            return
         for page in document.pages:
             if page.text_extraction_method != "pdftext" or not page.structure:
                 continue
-            apply = self.use_pdftext_reading_order or (
-                self.large_page_pdftext_order and self._is_large_page(page)
-            )
-            if not apply:
-                continue
 
             order = []
             last_pos = -1  # text-less blocks before any text sort to the top

marker/config/parser.py

@@ -95,24 +95,32 @@ def generate_config_dict(self) -> Dict[str, any]:
         config = {}
         output_dir = self.cli_options.get("output_dir", settings.OUTPUT_DIR)
         for k, v in self.cli_options.items():
-            if not v:
+            # None means "not provided". Explicit falsy values (False, 0) are
+            # real settings and must flow through - dropping them made it
+            # impossible to turn a default-True option off from the CLI.
+            if v is None:
                 continue
 
             match k:
                 case "debug":
-                    config["debug_pdf_images"] = True
-                    config["debug_layout_images"] = True
-                    config["debug_json"] = True
-                    config["debug_data_folder"] = output_dir
+                    if v:
+                        config["debug_pdf_images"] = True
+                        config["debug_layout_images"] = True
+                        config["debug_json"] = True
+                        config["debug_data_folder"] = output_dir
                 case "page_range":
-                    config["page_range"] = parse_range_str(v)
+                    if v:
+                        config["page_range"] = parse_range_str(v)
                 case "config_json":
-                    with open(v, "r", encoding="utf-8") as f:
-                        config.update(json.load(f))
+                    if v:
+                        with open(v, "r", encoding="utf-8") as f:
+                            config.update(json.load(f))
                 case "disable_multiprocessing":
-                    config["pdftext_workers"] = 1
+                    if v:
+                        config["pdftext_workers"] = 1
                 case "disable_image_extraction":
-                    config["extract_images"] = False
+                    if v:
+                        config["extract_images"] = False
                 case _:
                     config[k] = v
 

marker/processors/line_merge.py

@@ -12,38 +12,39 @@ class LineMergeProcessor(BaseProcessor):
     """
     A processor for merging inline math lines.
     """
-    block_types = (BlockTypes.Text, BlockTypes.TextInlineMath, BlockTypes.Caption, BlockTypes.Footnote, BlockTypes.SectionHeader)
+
+    block_types = (
+        BlockTypes.Text,
+        BlockTypes.TextInlineMath,
+        BlockTypes.Caption,
+        BlockTypes.Footnote,
+        BlockTypes.SectionHeader,
+    )
     min_merge_pct: Annotated[
-        float,
-        "The minimum percentage of intersection area to consider merging."
-    ] = .015
+        float, "The minimum percentage of intersection area to consider merging."
+    ] = 0.015
     block_expand_threshold: Annotated[
-        float,
-        "The percentage of the block width to expand the bounding box."
-    ] = .05
-    min_merge_ydist: Annotated[
-        float,
-        "The minimum y distance between lines to consider merging."
-    ] = 5
+        float, "The percentage of the block width to expand the bounding box."
+    ] = 0.05
     intersection_pct_threshold: Annotated[
         float,
-        "The total amount of intersection area concentrated in the max intersection block."
-    ] = .5
+        "The total amount of intersection area concentrated in the max intersection block.",
+    ] = 0.5
     vertical_overlap_pct_threshold: Annotated[
-        float,
-        "The minimum percentage of vertical overlap to consider merging."
-    ] = .8
-    use_llm: Annotated[
-        bool,
-        "Whether to use LLMs to improve accuracy."
-    ] = False
+        float, "The minimum percentage of vertical overlap to consider merging."
+    ] = 0.8
+    use_llm: Annotated[bool, "Whether to use LLMs to improve accuracy."] = False
 
     def __init__(self, config):
         super().__init__(config)
 
     def merge_lines(self, lines: List[Line], block: Block):
-        lines = [l for l in lines if l.polygon.width * 5 > l.polygon.height]  # Skip vertical lines
-        line_bboxes = [l.polygon.expand(self.block_expand_threshold, 0).bbox for l in lines]  # Expand horizontally
+        lines = [
+            ln for ln in lines if ln.polygon.width * 5 > ln.polygon.height
+        ]  # Skip vertical lines
+        line_bboxes = [
+            ln.polygon.expand(self.block_expand_threshold, 0).bbox for ln in lines
+        ]  # Expand horizontally
         intersections = matrix_intersection_area(line_bboxes, line_bboxes)
 
         merges = []
@@ -53,7 +54,9 @@ def merge_lines(self, lines: List[Line], block: Block):
             intersection_row[i] = 0  # Zero out the current idx
 
             if i < len(line_bboxes) - 1:
-                intersection_row[i+1] = 0 # Zero out the next idx, so we only evaluate merge from the left
+                intersection_row[i + 1] = (
+                    0  # Zero out the next idx, so we only evaluate merge from the left
+                )
 
             if len(merge) == 0:
                 merge.append(i)
@@ -74,14 +77,17 @@ def merge_lines(self, lines: List[Line], block: Block):
             vertical_overlap = max(0, vertical_overlap_end - vertical_overlap_start)
             vertical_overlap_pct = vertical_overlap / max(1, lines[i].polygon.height)
 
-            if all([
-                # Overlaps enough
-                intersection_pct >= self.min_merge_pct,
-                # Within same line
-                vertical_overlap_pct > self.vertical_overlap_pct_threshold,
-                # doesn't overlap with anything else
-                merge_intersection / total_intersection > self.intersection_pct_threshold
-            ]):
+            if all(
+                [
+                    # Overlaps enough
+                    intersection_pct >= self.min_merge_pct,
+                    # Within same line
+                    vertical_overlap_pct > self.vertical_overlap_pct_threshold,
+                    # doesn't overlap with anything else
+                    merge_intersection / total_intersection
+                    > self.intersection_pct_threshold,
+                ]
+            ):
                 merge.append(i)
             else:
                 merges.append(merge)
@@ -112,7 +118,6 @@ def merge_lines(self, lines: List[Line], block: Block):
             elif "math" not in line.formats:
                 line.formats.append("math")
 
-
     def __call__(self, document: Document):
         # Merging lines only needed for inline math
         if not self.use_llm:

marker/processors/llm/llm_table.py

@@ -28,10 +28,6 @@ class LLMTableProcessor(BaseLLMComplexBlockProcessor):
         int,
         "The maximum number of rows in a table to process with the LLM processor.  Beyond this will be skipped.",
     ] = 175
-    table_image_expansion_ratio: Annotated[
-        float,
-        "The ratio to expand the image by when cropping.",
-    ] = 0
     rotation_max_wh_ratio: Annotated[
         float,
         "The maximum width/height ratio for table cells for a table to be considered rotated.",

marker/processors/llm/llm_table_merge.py

@@ -25,10 +25,6 @@ class LLMTableMergeProcessor(BaseLLMComplexBlockProcessor):
         float,
         "The minimum height ratio relative to the page for the first table in a pair to be considered for merging.",
     ] = 0.6
-    table_start_threshold: Annotated[
-        float,
-        "The maximum percentage down the page the second table can start to be considered for merging.",
-    ] = 0.2
     vertical_table_height_threshold: Annotated[
         float, "The height tolerance for 2 adjacent tables to be merged into one."
     ] = 0.25