Merge pull request #417 from DavidS/claude/libvips-migration-plan-5CjpP

Migrate image pipeline from RMagick to libvips (1.2.0)

Author: DavidS

.github/workflows/tests.yaml

@@ -14,8 +14,8 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
-      - name: Install ImageMagick headers for rmagick
-        run: sudo apt-get update && sudo apt-get install -y libmagickwand-dev
+      - name: Install libvips for ruby-vips
+        run: sudo apt-get update && sudo apt-get install -y libvips42 libvips-dev libvips-tools
       - uses: ruby/setup-ruby@v1
         with:
           ruby-version: ${{ matrix.ruby }}
@@ -31,8 +31,8 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
-      - name: Install ImageMagick headers for rmagick
-        run: sudo apt-get update && sudo apt-get install -y libmagickwand-dev
+      - name: Install libvips for ruby-vips
+        run: sudo apt-get update && sudo apt-get install -y libvips42 libvips-dev libvips-tools
       - uses: ruby/setup-ruby@v1
         with:
           ruby-version: ${{ matrix.ruby }}
@@ -42,7 +42,22 @@ jobs:
         working-directory: spec/fixtures/test_site
         run: |
           bundle exec jekyll build --strict --trace --verbose
-          find _site -type f 
+          find _site -type f
           echo -e '\n===\n'
-          cat _site/gallery_one/index.html 
+          cat _site/gallery_one/index.html
           file _site/gallery_two/third/Morgenspaziergang-3.jpg_thumb.jpg
+      - name: Verify rendered output (dimensions + EXIF strip)
+        working-directory: spec/fixtures/test_site
+        run: |
+          set -euo pipefail
+          # gallery_two: max_size 600x400; source 1000x750 → 533x400
+          vipsheader _site/gallery_two/Frostig-001.jpg
+          test "$(vipsheader -f width  _site/gallery_two/Frostig-001.jpg)" = 533
+          test "$(vipsheader -f height _site/gallery_two/Frostig-001.jpg)" = 400
+          # 150x150 centre-cropped per-image thumb
+          test "$(vipsheader -f width  _site/gallery_two/Frostig-001.jpg_thumb.jpg)" = 150
+          test "$(vipsheader -f height _site/gallery_two/Frostig-001.jpg_thumb.jpg)" = 150
+          # EXIF must be stripped from every rendered output
+          ! vipsheader -a _site/gallery_two/Frostig-001.jpg | grep -i '^exif-'
+          ! vipsheader -a _site/gallery_two/Frostig-001.jpg_thumb.jpg | grep -i '^exif-'
+          ! vipsheader -a _site/gallery_one/2012-07-29-Eingeschlafen.jpg | grep -i '^exif-'

CHANGELOG.md

@@ -1,3 +1,31 @@
+# 1.2.0
+
+* **Breaking install change:** image processing switched from RMagick
+  to libvips (via [ruby-vips](https://github.com/libvips/ruby-vips)).
+  Install `libvips` (e.g. `apt install libvips42 libvips-dev` or
+  `brew install vips`) instead of ImageMagick / libmagickwand-dev. See
+  `docs/libvips-bench.md` for the perf comparison that motivated the
+  swap.
+* **Behaviour change:** EXIF Orientation is now honoured. Sources that
+  previously rendered sideways (Orientation 5–8) will now render
+  upright. Output JPEGs no longer carry the orientation tag.
+* **Behaviour change:** every rendered JPEG (full-size + thumbs) now
+  ships as a progressive JPEG with optimised Huffman tables, explicit
+  4:2:0 chroma subsampling, and all metadata stripped (`keep: :none`).
+  Thumbnails are encoded at Q80 and full-size renders honour the
+  collection-metadata `quality` value (default 85). Previously
+  thumbnails used RMagick's default encoder (baseline JPEG, ~Q75, EXIF
+  retained). The explicit `subsample_mode: :on` guards against a
+  silent ~25% file-size jump if a user bumps `quality` to ≥ 90 (where
+  libvips' `:auto` mode would otherwise switch to 4:4:4).
+* **Robustness:** image header reads now use `fail_on: :error`, so a
+  truncated or corrupt source JPEG aborts the build with a clear
+  error rather than silently rendering a half-grey output.
+* **Behaviour narrowing:** `max_size` values with ImageMagick trailing
+  flags other than `>` (i.e. `!`, `^`, `@`, `#`) are silently treated
+  as the plain `WxH` form. cheesy-gallery's own config never used
+  these flags; if you depended on them, file an issue.
+
 # 1.1.1
 
 * Address some deprecations warnings thanks to @pdxmph

CLAUDE.md

@@ -3,7 +3,7 @@
 ## Project
 
 `cheesy-gallery` — a Ruby gem providing a Jekyll plugin that turns
-directory trees of JPGs into navigable, RMagick-rendered photo galleries.
+directory trees of JPGs into navigable, libvips-rendered photo galleries.
 Powers <https://www.cheesy.at/fotos/>.
 
 ## Structure

README.md

@@ -2,6 +2,16 @@
 
 This is a jekyll photo gallery to manage large amounts of galleries and pictures. You can see the results at https://www.cheesy.at/fotos/.
 
+## Requirements
+
+Image rendering is done by [libvips](https://www.libvips.org/) via the
+[ruby-vips](https://github.com/libvips/ruby-vips) gem. Install libvips
+before installing the gem:
+
+- Debian / Ubuntu: `sudo apt install libvips42 libvips-dev`
+- macOS: `brew install vips`
+- Other distros: see the [libvips install docs](https://www.libvips.org/install.html)
+
 ## Installation
 
 Follow Jekyll's documentation on [how to install plugins](https://jekyllrb.com/docs/plugins/installation/) using "cheesy-gallery" as name for the gem and plugin.

cheesy-gallery.gemspec

@@ -39,5 +39,5 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency 'rubocop'
 
   spec.add_dependency 'jekyll', '~> 4.4'
-  spec.add_dependency 'rmagick', '>= 4', '< 6'
+  spec.add_dependency 'ruby-vips', '~> 2.2'
 end

docs/cache-analysis.md

@@ -1,10 +1,21 @@
 # Cache Mechanism Analysis (2026-05-11)
 
-How `cheesy-gallery` 1.1.1 caches work, how they sit on top of (and
+How `cheesy-gallery` 1.2.0 caches work, how they sit on top of (and
 sometimes around) Jekyll 4.4.1's own caching, where invalidation
 happens, and what happens once you put a `git-annex` worktree
 underneath it all.
 
+> **Note for readers of v1.2+:** this doc was originally written for the
+> RMagick-backed v1.1.x. Layers A–C and the invalidation model are
+> unchanged under the libvips backend. **Layer D** has been rewritten
+> below to describe libvips' operation cache instead of RMagick's
+> decoded pixel cache. Code snippets that mention `Magick::Image.ping`
+> or `Magick::ImageList.new` reflect the historical implementation;
+> the current call sites are `Vips::Image.new_from_file` (replacing
+> the ping) and `Vips::Image.thumbnail` (replacing the decode+resize
+> pair, now fused). The cache-spec spy targets are updated to match —
+> see `spec/cheesy/cache_spec.rb`.
+
 ## TL;DR
 
 - The plugin maintains **two named `Jekyll::Cache` instances** that
@@ -209,25 +220,47 @@ re-read at write time — geometry is only used to populate
 width=… height=…>` attributes. So Layer C does not gate I/O the way
 Layer B does; it gates the per-image `Magick::Image.ping` call.
 
-### 1.4 Layer D — RMagick's internal cache
+### 1.4 Layer D — libvips operation cache
 
-When we *do* render (Layer A and B both miss), `copy_file` in
-`base_image_file.rb:49-62` runs:
+When we *do* render (Layer A and B both miss), the subclass's
+`process_and_write` runs `Vips::Image.thumbnail(source_path, ...)`
+directly — libvips fuses decode + shrink-on-load + resize + (for
+thumbnails) centre-crop into a single operation. The base class no
+longer opens the source file itself; it just hands the source path
+to the subclass:
 
 ```ruby
-source = Magick::ImageList.new(path)
-begin
-  process_and_write(source, dest_path)
-ensure
-  source.destroy!
+# base_image_file.rb
+def copy_file(dest_path)
+  Jekyll.logger.debug 'Rendering:', dest_path
+  process_and_write(path, dest_path)
+  unless File.symlink?(dest_path)
+    File.utime(self.class.mtimes[path], self.class.mtimes[path], dest_path)
+  end
+  @@render_cache[render_cache_key(dest_path)] = true
 end
+
+# image_file.rb#process_and_write (full-size)
+img = Vips::Image.thumbnail(source_path, target_w, height: target_h,
+                            size: :down, crop: :none)
+img.write_to_file(dest_path, Q: @quality, interlace: true,
+                  strip: true, optimize_coding: true)
 ```
 
-`Magick::ImageList.new` decodes the JPEG into a pixel cache. RMagick
-keeps that decoded image in memory (and, depending on
-`MAGICK_TEMPORARY_PATH` and `MAGICK_DISK_LIMIT`, on disk too) until
-`destroy!` is called. The generator's *only* concession to this layer
-is the `collection.files.sort!` on `generator.rb:117`:
+What used to be "decode the entire JPEG into a pixel buffer and then
+resize" is now one library call that streams only the rows it needs
+(JPEG shrink-on-load at 1/2, 1/4, or 1/8 inside the codec, plus
+in-memory resize). There is **no separate decoded-pixel cache** to
+size, and no `destroy!` lifecycle — the `Vips::Image` is freed by GC.
+
+libvips does keep a small **operation cache** of recently-compiled
+operation graphs (default ~100 entries; tunable via
+`Vips.cache_set_max`). That's a process-local performance optimisation,
+not a correctness-affecting cache; it's transparent to our code and
+to the four-layer model above.
+
+The generator's `collection.files.sort!` on `generator.rb:117` still
+helps Layer D, but for a different reason now:
 
 ```ruby
 # sort files by source path, so that we have better cache hits when
@@ -238,12 +271,14 @@ is the `collection.files.sort!` on `generator.rb:117`:
 collection.files.sort! { |a, b| a.path <=> b.path }
 ```
 
-Sorting by source path keeps the full-size variant, the
-`*_thumb.jpg`, and (if applicable) the `*_index.jpg` for the same
-source next to each other in iteration order, so the underlying file
-data is more likely to be hot in the OS page cache when each variant
-opens it. There's no shared `Magick::ImageList` instance — each
-subclass opens, decodes, processes, and destroys independently.
+Sorting by source path keeps the full-size variant and its thumb(s)
+adjacent in iteration order. Under libvips this benefits **OS page
+cache locality** (each `Vips::Image.thumbnail` re-opens the file; the
+JPEG bytes are warm from the previous variant's open) and gives the
+libvips operation cache a better chance of reusing a previously-
+compiled graph. The TODO in the comment is now obsolete: there's no
+`ImageList` instance to share, because there's no separate decode
+step.
 
 This is the cache layer most affected by the choice of source-image
 storage (local FS vs. `git-annex`-resolved symlink vs. networked

docs/index.md

@@ -13,6 +13,9 @@ Internal documentation and reports for `cheesy-gallery`.
 - [Cache Mechanism Analysis](cache-analysis.md) — the four cache layers
   the plugin sits on top of, how they invalidate, and how `git-annex`
   source storage interacts with each layer (generated 2026-05-11).
+- [libvips Bench](libvips-bench.md) — perf comparison that motivated
+  the 1.2.0 RMagick → libvips migration; methodology, numbers across
+  small / DSLR workloads, and the gate decision (generated 2026-05-12).
 
 ## Working lists
 

docs/jekyll-api-review.md

@@ -222,28 +222,31 @@ is moot (Jekyll already covers it); one is still open.
 
 Estimated effort: ~half a day, all behind existing tests.
 
-### 3.2 Same architecture, swap RMagick for libvips
+### 3.2 Same architecture, swap RMagick for libvips _(landed in 1.2.0)_
 
-The numbers from `jekyll_picture_tag`'s migration and OpsLevel's blog
-suggest **5–10× build-time speedup** and **roughly an order of magnitude
-less RAM**. RMagick has also had recurring CVE / build-pain issues, while
-`ruby-vips` ships as a thin FFI wrapper.
-
-Two ways to land this:
-
-- **Drop-in via the [`image_processing`](https://github.com/janko/image_processing)
-  gem.** That gem exposes a single API over both libvips and
-  ImageMagick/MiniMagick, so we could keep RMagick as a fallback and
-  default to vips. Smallest behavioural risk.
-- **Direct `ruby-vips`.** Fewer dependencies, but a chunkier diff in
-  `image_file.rb` / `image_thumb.rb`.
+Originally a forward-looking option; landed in `cheesy-gallery 1.2.0`
+on top of PR #416 via the `claude/libvips-migration-plan-5CjpP` branch.
 
-Either way, the plugin's external interface (the collection metadata
-keys, the generated layout data) does not change. CI matrix needs to
-ensure libvips is installed (`apt install libvips`).
-
-Estimated effort: 1–2 days; bulk of the work is parity testing against
-existing fixtures.
+The numbers from `jekyll_picture_tag`'s migration and OpsLevel's blog
+suggested **5–10× build-time speedup** and **roughly an order of magnitude
+less RAM**. cheesy-gallery's empirical numbers turned out workload-
+dependent (see `docs/libvips-bench.md`): a **3.35× speedup at DSLR
+source sizes** (the production workload), and a slight regression for
+already-web-sized sources. RAM is roughly comparable to RMagick across
+both workloads, not the 10× public claim. The migration was justified
+on the DSLR speedup plus two non-perf wins: escape from RMagick's CVE /
+build-pain churn, and correct EXIF Orientation handling (latent bug
+under RMagick).
+
+The chosen path was **direct `ruby-vips`** (no `image_processing` gem,
+no RMagick fallback). The plugin's external interface (collection
+metadata keys, generated layout data) did not change. Subclass shape:
+`process_and_write` now takes `(source_path, dest_path)` and calls
+`Vips::Image.thumbnail` directly; the base class no longer opens the
+source file. CI installs `libvips42 libvips-dev libvips-tools` in
+place of `libmagickwand-dev`. See `CHANGELOG.md` for the user-facing
+notes (EXIF Orientation, thumb encoder settings, geometry-flag
+narrowing).
 
 ### 3.3 Split into two gems