Preliminary Checks
What happened?
Problem
When extracting raster images from PDFs, images placed with a horizontal flip transformation in the content stream are extracted inverted (mirrored left-right).
Example
- Page 15, Image 3 (the tobogán illustration):
- Full page render (
pg015_page.png): The slide ladder is on the right side, ramp descends left
- Extracted image (
pg015_im003.jpg): Ladder is on the left, ramp descends right
- Both are the same image, but one is horizontally flipped
Root Cause
In PDFs, image positioning and transformation is controlled by the CTM (Current Transformation Matrix) in the content stream. When an image is placed with a negative X scale (e.g., [-W 0 0 H tx+W ty]), it renders flipped horizontally.
- During full-page rendering (
page.toPixmap()): mupdf applies the CTM → flip is rendered correctly
- During raster extraction (
extractRasterImagesFromPdf): We read the raw XObject bytes directly from the PDF, without access to or application of the placement CTM → image is extracted in its original orientation (unflipped)
Why Only Some Images?
Only images placed with horizontal flip transformations in the PDF are affected. Most illustrations are placed without transformation, so their raw bytes match their visual appearance on the page. The tobogán was the only image on page 15 flipped—likely intentional design (e.g., composed to face text direction).
Solution
Extract the CTM from the content stream for each image, detect flip/rotation transforms, and apply them to the pixel data before saving the extracted image. This requires cross-referencing:
- Image XObject reference in content stream op
- The CTM at that point in the stream
- Apply any scale/rotation transforms to the extracted pixels
Related
- Code:
packages/pdf/src/extract.ts — extractRasterImagesFromPdf() function
- Stream recorder:
packages/pdf/src/page-stream-recorder.ts — has access to CTM
Impact
Component | Consequence
-- | --
LLM Processing | Receives image in wrong orientation; may make incorrect decisions about content/layout
Book Export | Illustrations appear flipped in final output
Storyboard Preview | Discrepancy between preview and actual rendering
User Experience | Confusion when exported book doesn't match what was shown during editing
Expected behavior
When a PDF contains a raster image placed with a horizontal flip transformation in the content stream: ✅ The extracted image (pg015_im003.jpg) should match the visual appearance on the page render (pg015_page.png) ✅ Both the full-page PNG and the cropped image should show the tobogán with: Ladder on the right side Ramp descending to the left ✅ When the extracted image is sent to the LLM for processing (image-cropping, segmentation, meaningfulness detection), it should be in the correct orientation ✅ Visual consistency across the entire pipeline: extract → storage → LLM → output
Actual behavior
Actual Behavior Currently, the extracted raster image is horizontally flipped compared to the page render: ❌ Page render (pg015_page.png): Shows tobogán with ladder on the right, ramp descending left ✓ Correct ❌ Extracted image (pg015_im003.jpg): Shows tobogán with ladder on the left, ramp descending right ✗ Inverted/Mirrored ❌ LLM receives: The inverted image without any indication that it's been transformed Image-cropping model sees a flipped version of what's in the book Segmentation and other steps process incorrect orientation ❌ Visual inconsistency: Preview (using page render) shows correct orientation Actual exported book (using extracted image) shows flipped orientation Storyboard displays inconsistent image
Steps to reproduce
Prerequisites
Have ADT Studio running locally with any PDF that contains raster images placed with horizontal flip transformations in the content stream. Most PDFs won't have this, but any professionally designed book may have intentionally flipped illustrations for layout purposes.
Step-by-Step Reproduction
- Open the Book
Launch ADT Studio. Navigate to Books and select any book from your collection. Click into the book to open it.
- Run Extract Step
Navigate to the Pipeline and locate the Extract step. Select a page range that you know contains images (or run the full book). Click "Run Extract" and wait for completion.
- Inspect Page Render
After extraction completes, navigate to the Image Filtering step. Select a page from the sidebar. View the full page render. Locate any illustration or diagram that has recognizable directional elements (a person facing a direction, an arrow, text orientation, etc.). Observe and mentally note the orientation of these elements.
- Inspect Extracted Image
Still in the Image Filtering view, scroll down to see the extracted images for that page. Find the image that corresponds to the illustration you just observed in the page render. View or download the extracted image.
- Compare Orientations
Compare the full page render with the extracted image. Check if any recognizable element that has a clear left/right or top/bottom orientation appears differently between the two images. Specifically, look for any illustration that appears horizontally flipped (left-right mirrored) in the extracted image compared to how it appears in the page render.
- Confirm the Issue
If the extracted image is horizontally flipped compared to the page render, the bug is confirmed. The full page rendering applies the CTM correctly and shows the image in the intended orientation. The extracted image, however, shows the raw XObject bytes without the flip transform applied, resulting in a mirrored appearance.
When This Occurs
This issue occurs only with images that have been placed in the PDF with a negative X scale in their transformation matrix. Most images in PDFs are placed without transformation, so they are unaffected. However, designers may intentionally flip images for layout purposes, and those images will exhibit this bug during extraction.
Screenshots and videos
No response
Version
v0.7.4-beta.4
What platform are you seeing the problem on?
Windows - x64
What area is this issue related to?
Pipeline — Extract
Relevant log output
Preliminary Checks
What happened?
Problem
When extracting raster images from PDFs, images placed with a horizontal flip transformation in the content stream are extracted inverted (mirrored left-right).
Example
pg015_page.png): The slide ladder is on the right side, ramp descends leftpg015_im003.jpg): Ladder is on the left, ramp descends rightRoot Cause
In PDFs, image positioning and transformation is controlled by the CTM (Current Transformation Matrix) in the content stream. When an image is placed with a negative X scale (e.g.,
[-W 0 0 H tx+W ty]), it renders flipped horizontally.page.toPixmap()): mupdf applies the CTM → flip is rendered correctlyextractRasterImagesFromPdf): We read the raw XObject bytes directly from the PDF, without access to or application of the placement CTM → image is extracted in its original orientation (unflipped)Why Only Some Images?
Only images placed with horizontal flip transformations in the PDF are affected. Most illustrations are placed without transformation, so their raw bytes match their visual appearance on the page. The tobogán was the only image on page 15 flipped—likely intentional design (e.g., composed to face text direction).
Solution
Extract the CTM from the content stream for each image, detect flip/rotation transforms, and apply them to the pixel data before saving the extracted image. This requires cross-referencing:
Related
packages/pdf/src/extract.ts—extractRasterImagesFromPdf()functionpackages/pdf/src/page-stream-recorder.ts— has access to CTMImpact
Expected behavior
When a PDF contains a raster image placed with a horizontal flip transformation in the content stream: ✅ The extracted image (pg015_im003.jpg) should match the visual appearance on the page render (pg015_page.png) ✅ Both the full-page PNG and the cropped image should show the tobogán with: Ladder on the right side Ramp descending to the left ✅ When the extracted image is sent to the LLM for processing (image-cropping, segmentation, meaningfulness detection), it should be in the correct orientation ✅ Visual consistency across the entire pipeline: extract → storage → LLM → output
Actual behavior
Actual Behavior Currently, the extracted raster image is horizontally flipped compared to the page render: ❌ Page render (pg015_page.png): Shows tobogán with ladder on the right, ramp descending left ✓ Correct ❌ Extracted image (pg015_im003.jpg): Shows tobogán with ladder on the left, ramp descending right ✗ Inverted/Mirrored ❌ LLM receives: The inverted image without any indication that it's been transformed Image-cropping model sees a flipped version of what's in the book Segmentation and other steps process incorrect orientation ❌ Visual inconsistency: Preview (using page render) shows correct orientation Actual exported book (using extracted image) shows flipped orientation Storyboard displays inconsistent image
Steps to reproduce
Prerequisites
Have ADT Studio running locally with any PDF that contains raster images placed with horizontal flip transformations in the content stream. Most PDFs won't have this, but any professionally designed book may have intentionally flipped illustrations for layout purposes.
Step-by-Step Reproduction
Launch ADT Studio. Navigate to Books and select any book from your collection. Click into the book to open it.
Navigate to the Pipeline and locate the Extract step. Select a page range that you know contains images (or run the full book). Click "Run Extract" and wait for completion.
After extraction completes, navigate to the Image Filtering step. Select a page from the sidebar. View the full page render. Locate any illustration or diagram that has recognizable directional elements (a person facing a direction, an arrow, text orientation, etc.). Observe and mentally note the orientation of these elements.
Still in the Image Filtering view, scroll down to see the extracted images for that page. Find the image that corresponds to the illustration you just observed in the page render. View or download the extracted image.
Compare the full page render with the extracted image. Check if any recognizable element that has a clear left/right or top/bottom orientation appears differently between the two images. Specifically, look for any illustration that appears horizontally flipped (left-right mirrored) in the extracted image compared to how it appears in the page render.
If the extracted image is horizontally flipped compared to the page render, the bug is confirmed. The full page rendering applies the CTM correctly and shows the image in the intended orientation. The extracted image, however, shows the raw XObject bytes without the flip transform applied, resulting in a mirrored appearance.
When This Occurs
This issue occurs only with images that have been placed in the PDF with a negative X scale in their transformation matrix. Most images in PDFs are placed without transformation, so they are unaffected. However, designers may intentionally flip images for layout purposes, and those images will exhibit this bug during extraction.
Screenshots and videos
No response
Version
v0.7.4-beta.4
What platform are you seeing the problem on?
Windows - x64
What area is this issue related to?
Pipeline — Extract
Relevant log output