fix: BitMaskedArray mask inversion and placeholder detection#4092
Open
henryiii wants to merge 4 commits into
Open
fix: BitMaskedArray mask inversion and placeholder detection#4092henryiii wants to merge 4 commits into
henryiii wants to merge 4 commits into
Conversation
henryiii
added a commit
that referenced
this pull request
Jun 10, 2026
Covers to_BitMaskedArray mask inversion, _unique consistency between BitMasked and ByteMasked option types, and _is_getitem_at_placeholder buffer check propagation. Assisted-by: ClaudeCode:claude-sonnet-4-6
❌ 1 Tests Failed:
View the top 1 failed test(s) by shortest run time
To view more test analytics, go to the Test Analytics Dashboard |
- Use bitwise `~` instead of `logical_not` in `to_BitMaskedArray` when inverting `valid_when` with the same `lsb_order`, matching the different-lsb_order branch and preventing silent data corruption. - Remove erroneous `out._content` unwrap in `BitMaskedArray._unique` so it matches `ByteMaskedArray._unique` semantics (both now delegate to `IndexedOptionArray64._unique` and return the result unchanged). - Fix `_is_getitem_at_placeholder` in `BitMaskedArray`, `ByteMaskedArray`, and `UnionArray` to check `.data` on the Index wrapper, consistent with `ListOffsetArray` and `IndexedArray` siblings. Assisted-by: ClaudeCode:claude-sonnet-4-6
Covers to_BitMaskedArray mask inversion, _unique consistency between BitMasked and ByteMasked option types, and _is_getitem_at_placeholder buffer check propagation. Assisted-by: ClaudeCode:claude-sonnet-4-6
Remove redundant round-trip, big-endian, and ByteMaskedArray tests that don't cover changed code paths. Remove placeholder tests that pass even without the fix. Rename file to use underscores per project convention. Two focused tests remain: one per distinct bug fixed. Assisted-by: ClaudeCode:claude-sonnet-4-6
a2418ab to
624fdb2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 AI text below 🤖
Summary
Three silent bugs fixed across
BitMaskedArray,ByteMaskedArray, andUnionArray:1.
BitMaskedArray.to_BitMaskedArray— bitmask inversion (data corruption)src/awkward/contents/bitmaskedarray.py:443usednplike.logical_not(self._mask.data)— a boolean byte-level NOT — to flipvalid_whenwhenlsb_orderis unchanged. This corrupts packed bitmasks: e.g.0b10011010becomes[False]rather than the correct bitwise inverse0b01100101. Changed to~self._mask.data(bitwise invert), matching thelsb_order-change branch at line 459.2.
BitMaskedArray._unique— spuriousout._contentunwrapsrc/awkward/contents/bitmaskedarray.py:659-668had a conditional that returnedout._contentwhennegaxis is not None, stripping the option-type wrapper from the result. The siblingByteMaskedArray._unique(bytemaskedarray.py) returnsoutunchanged. Removed the branch to matchByteMaskedArraysemantics.3.
_is_getitem_at_placeholderchecks Index wrapper instead of bufferThree methods checked
isinstance(self._mask/tags/index, PlaceholderArray)against the Index wrapper object, which is never aPlaceholderArray. SiblingsListOffsetArray(listoffsetarray.py:314) andIndexedArray(indexedarray.py:294) correctly check.data. Fixed in:src/awkward/contents/bitmaskedarray.py:499-502src/awkward/contents/bytemaskedarray.py:380-383src/awkward/contents/unionarray.py:587-591Test plan
tests/test_4092_bitmasked_fixes.py: one test per bug fixed (Fix 1: inversion, Fix 2: unique consistency)test_3033_flatten_bitmaskedarray.py,test_0127_tomask_operation.py,test_1260_simplify_masked_option_types.pyprek -a --quiet(clean)AI-assisted contribution: fixes identified and implemented with Claude Code (Anthropic), from an automated multi-agent review.