Release v1.1.0 (#255)

* Merge dev branch into release

Author: Erikpostt

.gitattributes

@@ -1,16 +1,8 @@
 * text=auto
-IMU_segment0001_meta.json filter=lfs diff=lfs merge=lfs -text
-IMU_segment0001_time.bin filter=lfs diff=lfs merge=lfs -text
-IMU_segment0001_values.bin filter=lfs diff=lfs merge=lfs -text
-IMU_segment0002_meta.json filter=lfs diff=lfs merge=lfs -text
-IMU_segment0002_time.bin filter=lfs diff=lfs merge=lfs -text
-IMU_segment0002_values.bin filter=lfs diff=lfs merge=lfs -text
-PPG_segment0001_meta.json filter=lfs diff=lfs merge=lfs -text
-PPG_segment0001_time.bin filter=lfs diff=lfs merge=lfs -text
-PPG_segment0001_values.bin filter=lfs diff=lfs merge=lfs -text
-PPG_segment0002_meta.json filter=lfs diff=lfs merge=lfs -text
-PPG_segment0002_time.bin filter=lfs diff=lfs merge=lfs -text
-PPG_segment0002_values.bin filter=lfs diff=lfs merge=lfs -text
+test_file.parquet filter=lfs diff=lfs merge=lfs -text
+example_data/**/*_time.bin filter=lfs diff=lfs merge=lfs -text
+example_data/**/*_values.bin filter=lfs diff=lfs merge=lfs -text
+example_data/**/*_meta.json filter=lfs diff=lfs merge=lfs -text
 
 # Normalize line endings
 *.py text eol=lf

.github/workflows/build-and-test.yml

@@ -55,10 +55,6 @@ jobs:
       - name: Run pytest
         run: poetry run pytest --maxfail=1 --disable-warnings -q
 
-      # Static type checks
-      - name: Type check with pytype
-        run: poetry run pytype .
-
       # Build the package
       - name: Build package
         run: poetry build

.gitignore

@@ -116,5 +116,18 @@ celerybeat.pid
 *.spec
 *.sage.py
 *.asv
+
 pip-log.txt
 pip-delete-this-directory.txt
+
+# =========================
+# Logs and databases
+# =========================
+*.log
+output*/
+
+# =========================
+# Generated TSDF files (from running notebooks/tests)
+# =========================
+example_data/**/segment*.json
+example_data/**/segment*.bin

.pre-commit-config.yaml

@@ -19,13 +19,6 @@ repos:
         pass_filenames: true
         types: [python]
 
-  - repo: https://github.com/PyCQA/isort
-    rev: 6.1.0
-    hooks:
-      - id: isort
-        args: ["--profile", "black"]
-        types: [python]
-
   - repo: https://github.com/astral-sh/ruff-pre-commit
     rev: v0.14.0
     hooks:

CHANGELOG.md

@@ -2,6 +2,49 @@
 
 <!--next-version-placeholder-->
 
+## v1.1.0 (04/02/2026)
+
+### Features
+- **Orchestrator Pipeline**: Added `orchestrator.py` with `run_paradigma()` function for complete end-to-end ParaDigMa analysis pipeline from data loading to aggregated results.
+- **Data Preparation**: Added `prepare_data.py` with automatic data preparation capabilities, including column mapping for flexible column names, watch orientation adjustment, and comprehensive validation.
+- **Centralized Loading**: Added `load.py` to centralize data loading capabilities across all pipelines.
+- **High-Level Pipeline Functions**: Added `run_gait_pipeline()`, `run_tremor_pipeline()`, and `run_pulse_rate_pipeline()` for simplified pipeline execution.
+- **Segment Distinction**: Added `GAIT_SEGMENT_NR` and `DATA_SEGMENT_NR` constants to distinguish between temporal gaps and gait bouts. `SEGMENT_NR` kept as deprecated alias for backward compatibility.
+- **Test Data**: Added Physilog Gait-Up test data to LFS.
+
+### Improvements
+- **Logging System**: Replaced `verbose` parameter with standard Python logging using `logging_level` and `custom_logger` parameters across all pipeline functions for better control and traceability.
+- **Data Segment Preservation**: Modified `quantify_arm_swing()` to preserve `data_segment_nr` when present in input data.
+- **Bug Fixes**: Fixed NaN propagation in resampling by interpolating NaN values before resampling (scipy cannot handle NaN in non-contiguous data).
+
+### Documentation
+- **New Tutorials**: Added `pipeline_orchestrator.ipynb` tutorial demonstrating end-to-end processing with logging control examples.
+- **New Guides**: Added comprehensive guides for data_input, installation, sensor_requirements, supported_devices, and validation.
+- **Readme**: Shortened README.md for readability and expanded detailed guides.
+- **PEP Standards**: Updated all docstrings and function signatures to PEP standards.
+- **Build System**: Added single-notebook build support with `--notebook` argument; automated style tag removal in documentation build pipeline.
+
+### Testing
+- Added minimal testing for new pipeline functionalities.
+- Added tests for segment column naming and backward compatibility.
+- Added tests to verify data segment tracking preservation.
+
+### Backward Compatibility
+- All existing code using `SEGMENT_NR` continues to work.
+- Previously used pipeline code remains functional.
+
+## v1.0.4 (11/11/2025)
+- Column names no longer have to be set to ParaDigMa standards, but can be flexibly adjusted (see data_preparation.ipynb tutorial for instructions).
+- Users can now change the tolerance threshold for determining when consecutive timestamps are contiguous using config.
+- The usage of accelerometry is now optional. Accelerometry can be used to detect motion artefacts that can be removed using Paradigma (see pulse_rate_analysis.ipynb for more details).
+- We also added instructions for how to scale PPG features using z-scoring.
+- Coefficient of variation (CoV) added as aggregation method for arm swing during gait.
+- Gait segment duration categories are no longer fixed.
+
+For developers
+* We added pre-commit hooks to ensure consistency in formatting and automate cleanup.
+* We also created two Python scripts for simplifying building and hosting docs, using poetry build-docs and poetry serve-docs.
+
 ## v1.0.3 (08/09/2025)
 - Added flexibility to let user specify gait segment duration categories.
 - Added the within-segment coefficient of variation to list of arm swing parameters.

CITATION.cff

@@ -5,6 +5,8 @@ message: >-
   If you use this software, please cite it using the
   metadata from this file.
 type: software
+version: 1.1.0
+date-released: '2026-02-04'
 authors:
 - given-names: Erik
   family-names: Post
@@ -95,18 +97,19 @@ abstract: >-
   package designed for processing and analyzing real-life wrist sensor data to
   extract digital measures of motor and non-motor signs of Parkinson's disease (PD).
 
-  Specifically, the toolbox is designed to process accelerometer, gyroscope and
-  photoplethysmography signals, collected during passive monitoring in daily life.
-  It contains three data processing pipelines: (1) arm swing during gait, (2) tremor,
-  and (3) pulse rate analysis. These pipelines are scientifically validated for their
-  use in persons with PD. Furthermore, the toolbox contains general functionalities
-  for signal processing and feature extraction, such as filtering, peak detection, and
-  spectral analysis.
+  The toolbox processes accelerometer, gyroscope and photoplethysmography signals
+  collected during passive monitoring in daily life. It contains three scientifically
+  validated data processing pipelines: (1) arm swing during gait, (2) tremor, and
+  (3) pulse rate analysis. An orchestrator function enables end-to-end processing from
+  raw data loading to aggregated measures, with automatic data preparation including
+  flexible column mapping and sensor orientation adjustment. The toolbox also provides
+  general functionalities for signal processing and feature extraction, such as
+  filtering, peak detection, and spectral analysis.
 
-  The toolbox is accompanied by a set of example scripts and notebooks for each
-  processing pipeline that demonstrate how to use the toolbox for extracting digital
-  measures. In addition, the toolbox is designed to be modular, enabling researchers
-  to easily extend the toolbox with new algorithms and functionalities.
+  ParaDigMa is accompanied by comprehensive documentation including tutorials,
+  installation guides, and API reference. The modular architecture enables researchers
+  to easily extend the toolbox with custom algorithms and functionalities while
+  maintaining compatibility with standardized data formats.
 
 keywords:
   - parkinson's disease

CONTRIBUTING.md

@@ -32,6 +32,58 @@ Documentation contributions are always welcome! You can contribute to:
 * Docstrings: In Python modules
 * Articles or blog posts
 
+#### Docstring Style Guide
+
+ParaDigMa follows NumPy/Napoleon docstring conventions with these type annotation guidelines:
+
+**Function Signatures:**
+- Use PEP 604 syntax: `str | Path` instead of `Union[str, Path]`
+- Use `X | None` instead of `Optional[X]`
+- Example:
+  ```python
+  def load_data(
+      path: str | Path,
+      config: Config | None = None
+  ) -> pd.DataFrame:
+  ```
+
+**Parameter Docstrings:**
+- Use natural language for types, not Python type syntax
+- For union types: `str or Path` not `Union[str, Path]`
+- For optional parameters: Add `, optional` suffix
+- For lists: `list of str` not `List[str]`
+- For dicts: `dict` not `Dict[str, int]`
+
+**Examples:**
+```python
+def example_function(
+    data_path: str | Path,
+    columns: List[str] | None = None,
+    config: Config | None = None,
+) -> pd.DataFrame:
+    """
+    Load and process data.
+
+    Parameters
+    ----------
+    data_path : str or Path
+        Path to data directory
+    columns : list of str, optional
+        Column names to load
+    config : Config, optional
+        Configuration object
+
+    Returns
+    -------
+    pd.DataFrame
+        Processed data
+    """
+```
+
+**Return Type Documentation:**
+- Use simple descriptions: `dict` or `DataFrame` not `Dict[str, pd.DataFrame]`
+- Add details in the description text below
+
 #### Workflow for notebooks and docs:
 1. Run and export notebooks:
 
@@ -65,6 +117,41 @@ If you are proposing a feature:
 ## Get Started!
 
 Ready to contribute? Here's how to set up `paradigma` locally:
+
+### Prerequisites
+
+1. **Install git-lfs** (required for example data):
+
+   **Windows:**
+   ```bash
+   choco install git-lfs
+   # Or download from https://git-lfs.com/
+   ```
+
+   **Linux:**
+   ```bash
+   # Ubuntu/Debian
+   sudo apt-get install git-lfs
+
+   # Fedora/RHEL
+   sudo dnf install git-lfs
+
+   # Arch Linux
+   sudo pacman -S git-lfs
+   ```
+
+   **macOS:**
+   ```bash
+   brew install git-lfs
+   ```
+
+2. **Enable git-lfs:**
+   ```bash
+   git lfs install
+   ```
+
+### Clone and Install
+
 1. Clone the repository:
 
 ```bash
@@ -78,13 +165,40 @@ cd paradigma
 poetry install
 ```
 
-3. Create a new branch for your work:
+3. **Verify example data setup:**
+
+   Check that example data was downloaded correctly (not git-lfs pointers):
+
+   **Linux/macOS:**
+   ```bash
+   ls -lh example_data/verily/ppg/*.bin
+   # Should show file sizes (100KB - several MB), not ~130 bytes
+
+   head -n 1 example_data/verily/ppg/PPG_segment0001_meta.json
+   # Should show JSON like {"file_name": ..., not "version https://git-lfs.github.com/spec/v1"
+   ```
+
+   **Windows PowerShell:**
+   ```powershell
+   Get-ChildItem example_data/verily/ppg/*.bin | Select-Object Name, Length
+   # Should show file sizes (100KB - several MB), not ~130 bytes
+
+   Get-Content example_data/verily/ppg/PPG_segment0001_meta.json -Head 1
+   # Should show JSON, not "version https://git-lfs.github.com/spec/v1"
+   ```
+
+   **If files are git-lfs pointers:**
+   ```bash
+   git lfs pull
+   ```
+
+4. Create a new branch for your work:
 
 ```bash
 git checkout -b name-of-your-bugfix-or-feature
 ```
 
-4. Make your changes and run the pre-commit hooks:
+5. Make your changes and run the pre-commit hooks:
 
 ```bash
 pip install pre-commit
@@ -93,16 +207,23 @@ pre-commit run --all-files
 ```
 
 This ensures code formatting (`black`), import sorting (`isort`), stripping notebook outputs, and other checks.
-These pre-commit hooks also run for changed and staged files when committing.
+These pre-commit hooks also run for changed files when committing.
 
-5. If contributing to docs, build and serve them locally to verify:
+6. If contributing to docs, build and serve them locally to verify:
 
 ```bash
 poetry run build-docs
 poetry run serve-docs
 ```
 
-6. Commit your changes and open a pull request.
+`build-docs` accepts the following arguments to speed up development:
+- `--notebook <filename>` - Build only a specific notebook (e.g., `--notebook pipeline_orchestrator.ipynb`)
+- `--dev` - Skip execution of notebooks
+- `--no-nbconvert` - Skip conversion of notebooks to markdown
+
+These options are useful when iterating on documentation changes without rebuilding everything.
+
+7. Commit your changes and open a pull request.
 
 ## Pull Request Guidelines