Skip to content

Feature/2026 04 18 bbrzycki implement simple native rawspec#43

Merged
bbrzycki merged 7 commits intomainfrom
feature/2026-04-18-bbrzycki-implement-simple-native-rawspec
Apr 21, 2026
Merged

Feature/2026 04 18 bbrzycki implement simple native rawspec#43
bbrzycki merged 7 commits intomainfrom
feature/2026-04-18-bbrzycki-implement-simple-native-rawspec

Conversation

@bbrzycki
Copy link
Copy Markdown
Owner

@bbrzycki bbrzycki commented Apr 21, 2026

This PR adds a native setigen.voltage reduction path for GUPPI RAW voltage outputs, reducing RAW stems or .raw files directly into .fil or .h5 filterbank products without requiring an external rawspec install for the common single-product workflow.

It also modernizes the package structure around pyproject.toml, expands Python version support to 3.10-3.14, and splits several large runtime modules into smaller private subsystems for frame construction, plotting, voltage backend orchestration, and RAW reduction.

User-Facing Changes

  • Adds stg.voltage.RawReductionSpec, PolarizationMode, reduce_raw(...), and reduce_raw_to_frame(...).
  • Adds the setigen-raw-reduce CLI for reducing RAW data from the shell.
  • Supports native RAW reduction to .fil and .h5.
  • Supports total power, full-pol, and full-Stokes output modes via pol_mode=1, 4, and -4.
  • Supports optional coarse-channel slicing with start_chan and num_chans.
  • Supports NumPy reduction by default, with optional CuPy acceleration.
  • Updates voltage documentation and README examples to show native reduction usage.
  • Keeps Frame as the inspection path for 2D total-power products via reduce_raw_to_frame(...).

Internal Changes

  • Refactors Frame internals into setigen._frame.
  • Refactors plot axis handling into setigen._plot.axes.
  • Refactors voltage antenna/backend logic into focused private modules under setigen.voltage._antenna and setigen.voltage._backend.
  • Adds a new setigen.voltage._reduction subsystem for RAW input parsing, decoding, fine channelization, metadata derivation, and streaming writers.
  • Adds shared typing helpers and explicit __all__ exports.
  • Moves packaging metadata to pyproject.toml; leaves setup.py as a minimal compatibility shim.
  • Updates CI to test Python 3.10 through 3.14 and install with .[test].
  • Adds contributor-facing CONTRIBUTING.md, ARCHITECTURE.md, and SCIENCE.md.

Testing

Added or expanded tests for:

  • RAW block decoding for 4-bit and 8-bit data.
  • Native reduction parity with existing helper behavior.
  • .fil and .h5 round trips.
  • total-power, full-pol, and full-Stokes products.
  • multi-file RAW stems.
  • CLI reduction.
  • signal detection after reduction and de-drifting.
  • voltage backend header generation, block planning, memory budgeting, and length-mode handling.
  • typing/docstring coverage for runtime modules.

- Implemented the `_reduction` module with functions for decoding RAW blocks, channelizing voltages, and writing output in filterbank formats.
- Created CLI for reducing GUPPI RAW files to filterbank formats (.fil and .h5).
- Added tests for the reduction functionality, ensuring correct decoding and channelization of voltage data.
- Introduced metadata handling for reduction processes, including support for multiple polarization modes.
- Enhanced error handling and validation for input parameters in the reduction process.
- Updated `raw_utils.py` to improve function signatures and add type hints for better clarity and maintainability.
- Enhanced `reduction.py` with type annotations and improved docstrings for better documentation and usability.
- Refactored `waterfall.py` to include type hints and improve function documentation.
- Introduced `_typing.py` to centralize type aliases and protocols for better code organization.
- Added tests for dedrift functionality to ensure proper error handling and behavior.
- Enhanced tests for voltage level calculations to verify scaling with SNR and observation length.
- Improved tests for reduction functions to ensure consistency and correctness in output.
- Conducted a docstring audit to ensure all functions and classes are properly documented and typed.
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 21, 2026

Codecov Report

❌ Patch coverage is 93.64496% with 121 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.22%. Comparing base (eea6844) to head (5379e21).

Files with missing lines Patch % Lines
setigen/_frame/io.py 71.01% 20 Missing ⚠️
setigen/voltage/_reduction/writers.py 88.59% 13 Missing ⚠️
setigen/voltage/_reduction/channelize.py 83.58% 11 Missing ⚠️
setigen/voltage/reduction.py 85.91% 10 Missing ⚠️
setigen/_frame/construction.py 92.85% 8 Missing ⚠️
setigen/_plot/axes.py 92.79% 8 Missing ⚠️
setigen/voltage/_reduction/input.py 91.48% 8 Missing ⚠️
setigen/voltage/_backend/pipeline.py 93.75% 7 Missing ⚠️
setigen/_frame/signal.py 93.87% 6 Missing ⚠️
setigen/voltage/_backend/headers.py 96.85% 4 Missing ⚠️
... and 16 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #43      +/-   ##
==========================================
+ Coverage   88.88%   92.22%   +3.33%     
==========================================
  Files          32       54      +22     
  Lines        2196     3189     +993     
==========================================
+ Hits         1952     2941     +989     
- Misses        244      248       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@bbrzycki bbrzycki merged commit 4cfaf00 into main Apr 21, 2026
12 checks passed
@bbrzycki bbrzycki deleted the feature/2026-04-18-bbrzycki-implement-simple-native-rawspec branch April 21, 2026 03:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant