Release v1.1.2#299
Closed
nienketimmermans wants to merge 119 commits into
Closed
Conversation
Align tremor pipeline
* Remove capitalization and rename gait to arm swing during gait * Change probability to predictions and update arrows in tremor pipeline * Change probability to predictions and update arrows in tremor pipeline --------- Co-authored-by: nienketimmermans <65012819+nienketimmermans@users.noreply.github.com>
The badge stayed at `3.10+` while the requirement changed to `3.11+`. A small update for consistency
Update Python requirement badge
Adding static pages for github pages to render
… into updates_pulse_rate
Updates pulse rate
Bump to v1.0.1
…e_rate fix refactoring
… into fix_docs_pulse_rate
Fix docs pulse rate
* Bump urllib3 to 2.6.1
* fix bug resampling * fix bug select_days function * json requires double quotes * undo changes json * incorporate modal tremor power in aggregate function * Incorporate suggestions * fix type * clarify variables * add tolerance to config * change preprocess_ppg_data too * add suggestions * run one segment for pulse rate only * fix typos * add suggestion * standardize columns in tremor analysis * pre-commit hooks * update poetry dependencies * update md files * change unit tremor power * update lock file * update markdown files * change printing of dictionaries * update md files
* Reframe index ToC, add dev option for building docs, add new preprints to readme
feat: add pipeline orchestrator with centralized logging and data preparation (#251) Implement end-to-end ParaDigMa analysis pipeline with automatic data preparation, centralized logging system, and improved documentation. Major changes: - Add orchestrator.py with run_paradigma() for complete analysis pipeline - Add prepare_data.py for automatic data preparation with column mapping and validation - Add load.py to centralize data loading capabilities - Replace verbose parameter with standard logging across all pipeline functions - Add GAIT_SEGMENT_NR and DATA_SEGMENT_NR constants to distinguish segment types - Add Physilog Gait-Up test data to LFS Pipeline improvements: - Add high-level functions: run_gait_pipeline(), run_tremor_pipeline(), run_pulse_rate_pipeline() - Fix NaN propagation in resampling by interpolating before resampling - Preserve data_segment_nr when present in quantify_arm_swing() - Add logging_level and custom_logger parameters for flexible log control Documentation: - Add pipeline_orchestrator tutorial for end-to-end processing - Add guides for data_input, installation, sensor_requirements, supported_devices, validation - Update docstrings to PEP standards - Add logging control section with examples - Add single-notebook build support with --notebook argument Testing: - Add tests for new pipeline functionalities - Add tests for segment column naming and preservation - Verify backward compatibility with SEGMENT_NR alias Build system: - Automate style tag removal in documentation build pipeline - Integrate clean_style_blocks() into build_docs.py Maintains backward compatibility with existing code.
* Update dependencies, bump to v1.1.0 * Update citation and changelog
* Solve conflicts
* Resolve merge conflicts with release
* Add unfiltered gait quantification to store both clean and all gait data
BREAKING CHANGE: run_gait_pipeline() now returns nested dicts instead of flat tuple
Previously, run_gait_pipeline() returned:
(quantified_df: DataFrame, metadata: dict)
Now returns:
({'filtered': df, 'unfiltered': df}, {'filtered': meta, 'unfiltered': meta})
This change allows analysis of both:
- Filtered (clean gait only): Arm swing during walking without other activities
- Unfiltered (all gait): All arm swing during any gait, regardless of quality
Changes:
- gait_pipeline.py: Call quantify_arm_swing() twice (filtered & unfiltered)
- gait_pipeline.py: Fix early returns to match new return structure
- orchestrator.py: Update result structure to handle nested gait dicts
- orchestrator.py: Separate Step 4 concatenation and Step 5 aggregation loops
- orchestrator.py: Aggregate both filtered and unfiltered quantifications
- orchestrator.py: Save both types of quantifications and aggregations
- test_gait_analysis.py: Update test assertions for new return structure
Motivation: Previous version only stored filtered quantification, losing
unfiltered data. This made it impossible to compare total gait duration
vs clean gait duration or analyze arm swing characteristics across all gait.
* Fix orchestrator to pass required segment_meta and segment_cats to aggregate_arm_swing_params
- Added parsing of segment_length_bins to segment_cats tuples
- Pass segment_meta from metadata dictionary to aggregation function
- Applies to both filtered and unfiltered gait aggregations
#267) * Clean up test infrastructure: remove unused test files, notebooks, and consolidate helpers to conftest.py
#269) * Fix save_intermediate parameter: use 'classification' for all prediction outputs and store predictions instead of features
* Bump version to 1.1.1 and update changelog
* Sync release into main
* modify spectrogram computation
…277) * Added option to run specific steps and return specific output. Updated docs accordingly
* test: Add datetime implementation test suite and verification * fix: Ignore unused start_dt in load_data_files * docs: Add backward compatibility analysis and comparison tools * Fix timestamp precision bug: add 1/fs to end_dt calculations Both combined and per_segment end_dt calculations now account for sample duration. Previously, end_dt used time_array.max() which represents the START of the last sample, not the END. Adding 1/fs (0.01s for 100Hz data) gives the correct end time. Fixes: segment durations now match duration_s values (e.g., segment 1 is 5.25s with end_dt 0.01s later) * Fix: Preserve timezone in quantification metadata when start_dt is passed with timezone * Ensure backward compatibility by including start_s and end_s EVEN in case of start_st
* fix: improve error handling and logging in gait pipeline - Add logging parameter to extract_arm_activity_features - Add logging parameter to quantify_arm_swing - Fix silent failure: raise ValueError on no windows instead of empty return - Replace print() with logger.warning() for proper log capture - Add context to errors: segment type, duration, data shape - Pass logger through function call chain in run_gait_pipeline When feature extraction fails or no data creates windows, now: - Error is logged to file with full context - Error includes segment/filter info and duration - Pipeline recognizes failure immediately - Users can trace errors back to specific subjects/files/segments
* fix: preserve per_segment metadata structure in gait pipeline orchestrator
* Fix: Use separate fields for segment categorization vs summation - duration_for_categorization_s: unfiltered parent segment duration - duration_s: actual segment data duration (filtered or unfiltered) Filtered arm swing of 5s in 25s unfiltered segment now correctly: - Categorizes into 20+s (based on parent unfiltered duration) - Sums only 5s to category total (actual filtered data) * Ensured gait categories are always scoped by unfiltered gait
* chore: update dependencies to patch security vulnerabilities - Update tornado to >=6.4.1 (fixes DoS and cookie injection vulnerabilities) - Update mistune to >=3.0.2 (fixes ReDoS in LINK_TITLE_RE) - Update pytest to >=7.4.4 (fixes tmpdir handling vulnerability) - Update jupyter-server to 2.18.2 (fixes CORS bypass, path traversal) - Update jupyterlab to 4.5.7 (fixes XSS vulnerabilities) - Update nbconvert to >=7.17.0 (fixes path traversal in cell attachments) - Update requests to >=2.32.6 (fixes insecure temp file reuse) - Update multiple dev dependencies to patch aiohttp issues Addresses 26 open security vulnerabilities from Dependabot scan * Reset jupyterlab-widgets to <= 3.0.15 because of installation errors for >= 3.0.16
…tment (#285) * feat: Add adaptive frequency bounds for variable sampling rates - Add sampling_frequency property with setter to IMUConfig - Implement _update_frequency_dependent_params() virtual method - Override in GaitConfig: auto-clamp spectrum_high_frequency and mfcc_high_frequency to Nyquist - Override in TremorConfig: auto-clamp fmax_peak_search and fmax_mfcc to Nyquist - Fixes issue where 50Hz/64Hz data caused 'out of bounds' errors in feature extraction This allows users to set sampling_frequency at any point and freq-dependent params adapt automatically, preventing Nyquist violations without manual config management. * fix: Only clamp MFCC bounds when they exceed Nyquist, preserve 25 Hz band
* prerelease adjustments * Update version date Updated version date for v1.1.2 and modified breaking changes section.
* Release v1.0.3 * Sync main to release (#233) * Incorporate modal tremor power in aggregate function * Add tolerance for contiguous data to config * Add pre-commit hooks and add Poetry scripts for simplifying dev process * Remove segment categorization (set to legacy) * Adjust pipelines to flexible column names and config updates * Add info on scale sensitivity and make accelerometer optional * Clarify installing pre-commit * Release v1.1.0 (#255) * Merge dev branch into release * Remove openmovement dependency (#256) * Remove openmovement dependency * Solve openmovement pypi conflict (#257) * Solve dependency conflicts and PyPi issue * Release v1.1.1 (#272) * Add unfiltered gait quantification to store both clean and all gait data * Fix orchestrator to pass required segment_meta and segment_cats to aggregate_arm_swing_params * Clean up test infrastructure: remove unused test files, notebooks, and consolidate helpers to conftest.py * Fix save_intermediate parameter: use 'classification' for all prediction outputs and store predictions instead of features * Bump version to 1.1.1 and update changelog * Remove tests/notebooks/ folder --------- Co-authored-by: Erik Post <57133568+Erikpostt@users.noreply.github.com>
* Release v1.0.3 * Sync main to release (#233) * Incorporate modal tremor power in aggregate function * Add tolerance for contiguous data to config * Add pre-commit hooks and add Poetry scripts for simplifying dev process * Remove segment categorization (set to legacy) * Adjust pipelines to flexible column names and config updates * Add info on scale sensitivity and make accelerometer optional * Clarify installing pre-commit * Release v1.1.0 (#255) * Merge dev branch into release * Remove openmovement dependency (#256) * Remove openmovement dependency * Solve openmovement pypi conflict (#257) * Solve dependency conflicts and PyPi issue * Release v1.1.1 (#272) * Add unfiltered gait quantification to store both clean and all gait data * Fix orchestrator to pass required segment_meta and segment_cats to aggregate_arm_swing_params * Clean up test infrastructure: remove unused test files, notebooks, and consolidate helpers to conftest.py * Fix save_intermediate parameter: use 'classification' for all prediction outputs and store predictions instead of features * Bump version to 1.1.1 and update changelog * Remove tests/notebooks/ folder * fix unintended changes --------- Co-authored-by: Erik Post <57133568+Erikpostt@users.noreply.github.com>
Contributor
|
@copilot resolve the merge conflicts in this pull request |
|
@Erikpostt I can’t apply these changes because the pull request’s branch is the repository’s default branch, which I’m not allowed to push to directly. You can ask me to create a new pull request with the requested changes instead and I'll open a separate PR on a new branch. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release v1.1.2
Release date: 12/06/2026
Description
This PR synchronizes the
mainbranch intoreleaseto prepare for the v1.1.2 release of ParaDigMa.Changes
Breaking Changes
run_gait_pipeline(),run_tremor_pipeline(),run_pulse_rate_pipeline()) now return a unified dictionary structure instead of tuples/DataFrames:(quantified_df_dict: dict, metadata_dict: dict)(from v1.1.1)DataFrame'quantification': Computed results (dict for gait, DataFrame for others)'metadata': Metadata dict (gait only)'preprocessing','classification': Intermediate results (if requested viareturn_intermediate)'_steps_executed': List of steps that were actually executed'_error': Error message (None if successful)Features
Selective pipeline step execution: New parameters
run_stepsandreturn_intermediatein all pipeline functions and orchestrator:run_steps: Which processing steps to execute ('preprocessing', 'classification', 'quantification', 'aggregation')return_intermediate: Which intermediate results to include in return dict (independent of disk storage)run_paradigma()orchestratorrun_steps=['preprocessing', 'classification']to stop after classification without computing quantificationAdaptive sampling frequency detection: orchestrator now automatically detects data sampling frequency and adjusts frequency-dependent parameters accordingly. It is no longer necessary to manually set the sampling frequency to 100 Hz
Independent save/return control:
save_intermediateandreturn_intermediatenow work independently:Orchestrator updates:
run_paradigma()now supports selective step execution and handles unified dict returns from all pipelines:Improvements
_errorfield in all pipeline function returns.Checklist