Skip to content

00-scope-and-definitions.md

Latest commit

 

History

History
157 lines (110 loc) · 6.08 KB

00-scope-and-definitions.md

File metadata and controls

157 lines (110 loc) · 6.08 KB
Raw

Scope and definitions

This repository is about table detection: locating table regions on a page/image.

If you’re looking for table structure recognition (rows/columns/cells) or table extraction (end-to-end to CSV/HTML), those are downstream tasks. We reference them only to clarify interfaces and evaluation boundaries.

What is table detection?

Table detection is the task of finding where tables are in a document page or image.

Typical outputs:

  • Bounding boxes (x, y, w, h) for each table
  • Polygons / masks for each table (less common, helpful for curved/perspective tables)
  • Optional: table confidence score per region
  • Optional: table class (e.g., bordered vs borderless) — only if your dataset defines it

Input types:

  • Raster images: scans, photos, screenshots
  • PDF pages: either rasterized or parsed/rendered into images for ML inference

What this repo does and does not cover

In scope ✅

  • Detecting table regions in:
    • Documents (PDFs, scanned pages, reports, papers, invoices)
    • In-the-wild imagery (camera photos, web screenshots, whiteboards, slides)
  • Model families and practical tradeoffs for detection
  • Datasets that provide table region annotations (boxes/polygons)
  • Evaluation metrics and benchmark protocols for detection
  • Open-source and commercial solutions as they relate to detection
  • How VLMs (Vision-Language Models) can help with detection (especially open-vocabulary detection and weak labeling)

Out of scope ❌ (but referenced for context)

  • Table structure recognition (TSR): cells/rows/columns, spanning, logical structure
  • OCR: recognizing text inside the table
  • Table extraction: producing CSV/HTML/JSON from tables end-to-end
  • Document understanding tasks like QA, key-value extraction, or RAG pipelines
  • Complex post-processing rules for structure reconstruction

Related tasks and how they connect

Table detection in a document pipeline

Detection “flavors” you will see in practice

A) Document layout detection (multi-class)

A “layout detector” predicts multiple region types (text blocks, titles, figures, tables, etc.).

  • Pros: strong on PDFs and structured documents; learns page context
  • Cons: can be weaker on weird table appearances outside doc domain

B) Table-only object detection

A detector is trained specifically to detect tables (sometimes plus related classes).

  • Pros: simple, often strong if domain matches training data
  • Cons: generalization depends heavily on dataset diversity

C) Instance segmentation / polygon detection

Predicts a mask/polygon for the table region.

  • Pros: useful for rotated/curved/perspective tables
  • Cons: heavier training/inference, more annotation cost

D) Open-vocabulary / VLM-grounded detection

Promptable detection (“find tables”) sometimes with weak supervision or grounding.

  • Pros: great for bootstrapping labels and edge cases
  • Cons: consistency, latency, and cost can be issues versus dedicated detectors

Output formats and coordinate conventions

Bounding boxes

Common formats:

  • XYXY: (x_min, y_min, x_max, y_max)
  • XYWH: (x, y, width, height) Coordinates can be:
  • absolute pixels (most common for raster)
  • normalized [0,1] relative to image size

Polygons / masks

  • Polygon: list of points [(x1,y1), (x2,y2), ...]
  • Mask: binary image (same size as input)

Multi-page documents

  • The unit of inference is usually per page
  • If you want “multi-page tables,” treat that as a higher-level grouping problem (out of scope)

What counts as a “table”?

This is surprisingly dataset-dependent. For this repo:

A table is a region that primarily encodes information in a grid-like, tabular layout (rows/columns), with or without headers, regardless of whether grid lines are visible.

Common edge cases:

  • Borderless tables (aligned text, whitespace separators)
  • Tables with merged cells (still tables)
  • Tables embedded in figures (varies by dataset)
  • Lists that look table-like (dataset-dependent)
  • Calendar layouts (sometimes labeled as tables, sometimes not)
  • Forms (not tables unless explicitly annotated as tables)

When comparing tools/models, always check the dataset definition of “table.”

Common failure modes (what to watch for)

  • Borderless tables detected as text blocks
  • Nested tables or “table inside a figure”
  • Small tables missed due to resizing/downsampling
  • Rotated / perspective tables in photos
  • Tables spanning columns in multi-column layouts
  • Partial detection (detecting only header or only body)
  • Over-detection (false positives on aligned text or charts)
  • Split/merge errors:
    • one table predicted as multiple boxes
    • multiple tables merged into one box

How we’ll evaluate “good detection”

Detection quality depends on the intended downstream use:

  • High recall if missing a table is costly (archiving, compliance, extraction pipelines)
  • High precision if false positives are costly (human review, automated export to structured systems)
  • Stable geometry if you need consistent cropping for TSR/OCR

We cover metrics and recommended protocols in:

Terminology used in this repo

  • Document tables: tables from PDFs/scans (papers, invoices, reports)
  • Tables in the wild: camera photos, screenshots, unconstrained imagery
  • Detector: model that predicts table regions
  • Layout model: detector that predicts multiple region categories, including tables
  • Post-processing: NMS, thresholding, merging/splitting heuristics after inference
  • TSR: table structure recognition (rows/columns/cells)

Common confusion: "Table detection" finds where tables are (regions/boxes). "Table recognition" or "table structure recognition" finds what's inside (cells, rows, columns). Many tools do both, but they are distinct tasks with different evaluation metrics.