Conversation
kujaku11
commented
Feb 10, 2026
kujaku11
commented
Collaborator
- Bug fixes
- Adding option to open file in swmr
- Updating mth5 validator
- adding publish action
Improved the calculation of the number of samples during channel merge to robustly handle different timedelta types. Added a test to verify that merging channels with gaps produces the correct data size, sample rate, and component.
Added type annotations and improved docstrings for clarity and consistency in zen_tools.py. Enhanced error handling and logging, updated function signatures, and improved schedule file writing logic. Adjusted related tests to match new error types and expectations.
Updated type annotations in z3d_header.py and z3d_metadata.py to use the PEP 604 union syntax (e.g., 'str | None' instead of 'Optional[str]' or 'Union[str, None]'). This modernizes the codebase and improves readability by leveraging Python 3.10+ features.
Replaced usage of typing.Union with the PEP 604 (|) union syntax for type hints in NIMSCollection class methods and docstrings. This modernizes the code and improves readability.
Added a new documentation page summarizing the MTH5 class's attributes and methods, including usage examples. Updated the documentation index to include the new summary page.
Added a debug log message when no coil calibrations are loaded in CoilResponse. Updated test to check for KeyError with coil number 2172. Added profiling output files for various test cases.
Introduces list_reports and remove_report methods to the ReportsGroup class for managing stored reports and images. Refactors add_report and get_report for improved metadata handling and logging. Adds a comprehensive pytest suite to test report addition, retrieval, listing, and removal functionalities.
Added a new Jupyter notebook example demonstrating how to add and extract reports (text and image files) to/from an HDF5 group using mth5. Included example text and image files. Also updated `mth5/groups/reports.py` to support these examples.
Replaces usage of undefined 'nm_url' with 'self.national_map_url' in the debug log statement within AsciiMetadata. This ensures the correct URL is logged when an HTTPError occurs.
Introduce a full MTH5 file validator and CLI: new mth5/utils/mth5_validator.py (MTH5Validator, ValidationResults, messages, metadata/structure/data checks) and mth5/utils/cli.py (mth5-cli validate). Add documentation and quickstart: docs/source/VALIDATOR_README.md, VALIDATOR_QUICKSTART.md, VALIDATOR_IMPLEMENTATION.md. Include example usage script examples/validator_examples.py and a demo test tests/utils/test_validator_demo.py. Update mth5/utils/__init__.py to avoid circular imports and update pyproject.toml (added CLI entry point). These changes provide programmatic and CLI validation, docs, examples, and basic tests.
Add a large set of sphinx-autoapi-generated RST files for the mth5 package (clients, data, groups, io, processing, timeseries, utils, etc.), and update docs/conf.py and docs/index.rst to include the new API reference. Remove stale profiling artifacts under tests/prof and add a new unit test tests/test_mth5_validator.py. These changes enable automated API documentation and tidy up test artifacts.
Add cross-platform build tooling to produce a standalone mth5-validator executable: build_executable.py (recommended, handles PyInstaller install, cleaning, build and test), platform wrappers build_executable.bat and build_executable.sh, and a detailed BUILD_EXECUTABLE_README.md. The PyInstaller configuration includes hidden imports for h5py, metadata collection for mth5/mt_metadata, and exclusions to reduce bundle size; the entry point is mth5/utils/cli.py. Remove a large set of auto-generated Sphinx autoapi .rst files under docs/autoapi, cleaning out previously generated API reference files (likely to be regenerated by a different process or to reduce checked-in generated docs).
Replace @requires_test_data decorators with runtime has_test_data checks and pytest.skip calls in Metronix tests so they gracefully skip when real test data is unavailable. Rename tests/test_mth5_validator.py -> tests/utils/test_mth5_validator.py, add import of Station from mt_metadata.timeseries and construct station metadata via Station(). Remove the demo validator script tests/utils/test_validator_demo.py. These changes improve test robustness and reorganize validator tests.
This is needed for python versions 3.14+, which removes setuptools from the standard library.
Introduce Single Writer Multiple Reader (SWMR) support and related documentation/examples. - mth5/mth5.py: add single_writer_multiple_reader parameter to open_mth5 (aliased to kwargs.swmr), enforce libver='latest', validate incompatible modes (e.g. 'w', 'x'), require existing file for SWMR, set swmr flag for readers and activate h5py SWMR writer mode after opening for writers. Add flush() method for writers to make data visible to readers, integrate SWMR-aware behavior into close_mth5, and add is_swmr_mode() helper. - docs/index.rst: register new SWMR guide in the table of contents. - docs/source/SWMR_GUIDE.md: new comprehensive guide covering usage, gotchas, examples, best practices and troubleshooting for SWMR workflows. - examples/: add test_swmr_basic.py (unit-style script for SWMR scenarios) and test_swmr_concurrent.py (multiprocessing writer + readers demo). These changes enable concurrent read access during real-time/append-only writes and provide guidance and examples for correct usage (flush, modes, file creation requirements).
Add comprehensive SWMR documentation and examples (migration, basic and concurrent) and wire them into docs/index. Add a new SWMR-focused test suite. Update MTH5 to default libver='latest' (and enforce it when SWMR is requested) to ensure SWMR compatibility; adjust close/flush behavior and summary updates to be safe for SWMR writers. Also add a pytest "slow" marker in pyproject.toml.
CI: run pytest only for fast tests by adding -k "not slow" to the workflow. Tests: extensive updates to tests/test_mth5_open_swmr.py to respect HDF5 SWMR limitations. Added top-of-file guidance about what SWMR allows (read/append-only, no group/dataset creation) and test run instructions. Skipped/updated tests that attempted to create datasets or groups in SWMR mode (skip add-data test and pre-create structure where needed). Marked long-running tests as slow, reduced reader count for speed, and skipped a fragile multiprocessing test. Adjusted assertions and fixtures to use existing basic test files and changed performance/flush tests to avoid creating data under SWMR (only flush operations). Overall changes make tests safer, faster by default, and SWMR-compliant.
Add a GitHub Actions workflow (.github/workflows/publish.yml) to bump versions, generate changelogs, create releases and publish wheels to TestPyPI/PyPI on merged PRs (supports dry-run and label-based bump types). Remove legacy executable build artifacts and helper scripts (BUILD_EXECUTABLE_README.md, build_executable.bat, build_executable.sh) and rename/refactor build_executable.py -> build_standalone_validator.py with improved dependency checks, cleaning, and PyInstaller args for a lightweight standalone validator. Add mth5_validator_standalone.py: a new h5py-based, minimal MTH5 file validator CLI intended for building small (~20–30 MB) executables and validating file structure without the full mth5 stack. Overall this replaces the old full mth5 packaging approach with a streamlined standalone validator and automated publish pipeline.
Update CI to install git-cliff via Rust/Cargo (install cargo and run `cargo install git-cliff`) instead of using apt's git-cliff package and remove the prior pip installs. Add a new cliff.toml to configure changelog generation with a header, body, conventional commits enabled, and filter_unconventional set to false. This ensures the workflow uses the Cargo-distributed git-cliff and provides project-specific changelog settings.
Adding Single write multiple reader
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #317 +/- ##
==========================================
- Coverage 81.82% 81.12% -0.71%
==========================================
Files 223 232 +9
Lines 46520 47934 +1414
==========================================
+ Hits 38066 38887 +821
- Misses 8454 9047 +593
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.