Skip to content

fix: typetracer correctness issues affecting dask-awkward#4100

Open
henryiii wants to merge 3 commits into
mainfrom
henryiii/fix-typetracer
Open

fix: typetracer correctness issues affecting dask-awkward#4100
henryiii wants to merge 3 commits into
mainfrom
henryiii/fix-typetracer

Conversation

@henryiii

Copy link
Copy Markdown
Member

🤖 AI text below 🤖

This PR fixes several typetracer correctness issues, most of which affect dask-awkward (which runs awkward operations on shape-only typetracer arrays). They were surfaced by an automated multi-agent code review (Claude Code) and each was verified before fixing.

Fixes

  • TypeTracer.min/max/sum(axis=None) infinite recursion (_nplikes/typetracer.py): min recursed with axis=axis (still None) instead of axis=0, causing a RecursionError; max/sum delegated to it. Additionally, sum now reports NumPy's promoted output dtype (e.g. bool/int32 → platform int) rather than preserving the input dtype, matching the concrete backends.
  • TypeTracer.stack wrong shape/dtype (_nplikes/typetracer.py): it treated the prototype tuple (0,) * ndim as data for empty_like, and normalized negative axes over the input ndim instead of the ndim + 1 output. Now the result shape is built symbolically (inserting len(arrays) at the normalized axis) and the dtype uses numpy.result_type. Reachable from _broadcasting.py.
  • searchsorted dtype (_nplikes/typetracer.py): returned the input array's dtype; it returns insertion indices, so it now reports intp, matching np.searchsorted and the convention used by nonzero.
  • asarray dtype handling (_nplikes/typetracer.py): for non-TypeTracerArray inputs it silently dropped the requested dtype and mis-fired the copy=False dtype check when dtype is None. It now honors the requested dtype and only raises for copy=False when an incompatible dtype is explicitly requested.
  • reshape with zero-size dimensions (_nplikes/typetracer.py): rejected any 0, but zero-size reshapes are valid in NumPy and the concrete path. Now permitted (with the -1 placeholder handled consistently for zero size).
  • __repr__ mutating the report (_nplikes/typetracer.py, _nplikes/placeholder.py, _nplikes/virtual.py): __repr__ read the touching .shape property, so merely printing an array mutated the TypeTracerReport. Now uses self._shape, and the impossible if shape is None dead branch is removed in all three.
  • CuPy reducers ignoring keepdims (_nplikes/cupy.py): the five overridden reducers (all/any/min/sum/max) accepted keepdims but never forwarded it. They now forward it, and the cupy#3819 .item() workaround is applied only for axis=None without keepdims so that axis=None, keepdims=True returns the 1×…×1 array NumPy produces.
  • ak.pad_none on 1D typetracer arrays (contents/numpyarray.py): _pad_none with clip=False did target < self.length without an unknown_length guard, raising TypeError under dask-awkward. Now uses the same guard pattern as content.py.
  • ak.combinations(axis=0) on typetracer (contents/content.py): _combinations_axis0 checked size is None, but unknown lengths are unknown_length, never None, so it raised TypeError. Now checks against unknown_length.
  • Dead-code removal (_nplikes/typetracer.py): removed a verified-unreachable integer bounds-check in __getitem__ (the index is always an unknown scalar after asarray), two dead is_unknown_scalar(length_scalar) branches in derive_slice_for_length (only reached when the length is known), and a dead hasattr(x, "shape") loop in broadcast_arrays (every element is asserted to be a TypeTracerArray).

Testing

A regression test file exercises all of the above on typetracer layouts (using to_typetracer(forget_length=True) for the high-level ak.pad_none/ak.combinations cases). The existing typetracer suites and the relevant operation suites pass.

The CuPy change is untested locally (no GPU available); it was made by inspection, mirroring the NumPy base class.

AI assistance

This PR was prepared with Claude Code via an automated multi-agent review. All findings were verified before fixing.

🤖 Generated with Claude Code

@codecov

codecov Bot commented Jun 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.50000% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.38%. Comparing base (712dac0) to head (ca3df4a).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/awkward/_nplikes/typetracer.py 83.33% 7 Missing ⚠️
Additional details and impacted files
Files with missing lines Coverage Δ
src/awkward/_nplikes/cupy.py 91.45% <100.00%> (ø)
src/awkward/_nplikes/placeholder.py 65.44% <100.00%> (+0.22%) ⬆️
src/awkward/_nplikes/virtual.py 93.30% <100.00%> (+0.41%) ⬆️
src/awkward/contents/content.py 77.61% <100.00%> (+0.16%) ⬆️
src/awkward/contents/numpyarray.py 91.56% <100.00%> (+0.17%) ⬆️
src/awkward/_nplikes/typetracer.py 77.24% <83.33%> (+1.63%) ⬆️

henryiii added 3 commits June 11, 2026 15:40
Fixes several typetracer correctness issues, mostly affecting
dask-awkward, found via an automated multi-agent review:

- typetracer min/max/sum(axis=None) recursed with axis=None instead of
  axis=0, causing a RecursionError; sum now reports NumPy's promoted
  dtype (e.g. bool/int32 -> platform int).
- typetracer.stack built the result shape incorrectly (treated the
  prototype tuple as data, and normalized negative axis over the input
  ndim rather than ndim+1); now builds the shape symbolically and uses
  numpy.result_type for the dtype.
- searchsorted returned the input dtype instead of intp insertion
  indices.
- asarray now honors the requested dtype for non-TypeTracerArray inputs
  and only rejects copy=False when an incompatible dtype is requested.
- reshape now permits zero-size dimensions, matching NumPy and the
  concrete backends.
- TypeTracerArray.__repr__ no longer reads the touching .shape property
  (printing mutated the report); same dead None-shape branch removed in
  PlaceholderArray and VirtualNDArray.
- cupy reducers (all/any/min/sum/max) now forward keepdims, preserving
  the cupy#3819 .item() workaround only for axis=None without keepdims.
- NumpyArray._pad_none and Content._combinations_axis0 now guard against
  unknown_length, fixing ak.pad_none and ak.combinations(axis=0) on
  typetracer layouts.
- Removed verified-dead branches in __getitem__, derive_slice_for_length,
  and broadcast_arrays.

Assisted-by: ClaudeCode:claude-opus-4.8
Assisted-by: ClaudeCode:claude-opus-4.8
Keep one focused test per fix and drop comments that merely restate
the adjacent code.

Assisted-by: ClaudeCode:claude-opus-4-8
@henryiii henryiii force-pushed the henryiii/fix-typetracer branch from c039be1 to ca3df4a Compare June 11, 2026 19:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant