Add deprecations merge command to combine parallel CI shards#177
Open
Add deprecations merge command to combine parallel CI shards#177
Conversation
Introduces ShardMerger class that deep-merges shard files written by parallel CI nodes into a single canonical shitlist file. Supports --delete-shards, --next, and --path flags. Fixes the inject(:merge) bug from PR #83 where overlapping buckets lost data.
Member
Author
|
If you want to test locally, here you have some fake deprecation warnings shitlist.zip |
arielj
reviewed
Apr 6, 2026
- Remove --path flag (unrelated to merge, belongs in its own PR) - Move delete_shards to ShardMerger initializer - Change "test files" to "buckets" in merge output - Add error handling for file read and JSON parse errors - Fix delegation test to verify ShardMerger receives correct params - Use .json extension in tempfiles for realistic shard_glob testing - Add spec for invalid JSON error handling
The behavior is fully covered in shard_merger_spec.rb.
arielj
reviewed
Apr 7, 2026
- Move require_relative to top of file with other requires - Unify all modes into single case statement instead of early-return - Include merge mode in missing-mode error message - Warn and return early when directory or shards not found instead of silently writing empty JSON - Hardcode .json extension in shard glob since only JSON is supported
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a
deprecations mergeCLI command andDeprecationTracker::ShardMergerclass to combine shard files produced by parallel CI nodes into a single canonical shitlist file.This is the companion to #176 (parallel CI support). When each CI node writes its own shard (e.g.
deprecation_warning.shitlist.node-0.json), a fan-in step is needed to merge them before the compare phase.Changes
DeprecationTracker::ShardMerger— new class inlib/deprecation_tracker/shard_merger.rbwith deep-merge logic (concatenates arrays for overlapping buckets), fixing the data loss bug from Adds support to print info for multiple files. #83 whereinject(:merge)silently dropped warnings when multiple shards shared the same keys.deprecations mergeCLI subcommand — with--delete-shardsto clean up after merging and--nextfor the next Rails shitlist.DeprecationTracker.merge_shards— convenience class method that delegates toShardMerger.spec/shard_merger_spec.rbcovering: multiple shards, overlapping buckets, no shards, single shard, delete/preserve shards, sort order, and invalid JSON.Example CI workflow
Test plan
bundle exec rspec spec/deprecation_tracker_spec.rb spec/shard_merger_spec.rb— 42 examples, 0 failures--delete-shardsremoves shard files, canonical file remains