Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
46b4bb8
Update README.md
van-dai-vuong Jun 4, 2025
ecd4883
Update README.md
van-dai-vuong Jun 4, 2025
1433750
Update README.md
van-dai-vuong Jun 4, 2025
a75e086
Update README.md
van-dai-vuong Jun 4, 2025
1eb9791
Update README.md
van-dai-vuong Jun 4, 2025
6a0204b
Update README.md
van-dai-vuong Jun 4, 2025
7c5d099
Update pyproject.toml
van-dai-vuong Jun 4, 2025
31fda70
Merge branch 'feature/version_up' of https://github.com/Bayes-Works/c…
van-dai-vuong Jun 4, 2025
a8103e2
Update conf.py
van-dai-vuong Jun 4, 2025
92ab96e
Update Canari_example.png
van-dai-vuong Jun 4, 2025
258e0d7
Update README.md
van-dai-vuong Jun 4, 2025
8bc72ce
Update index.rst
van-dai-vuong Jun 4, 2025
3a88cdf
Update build_wheels.yml
van-dai-vuong Jun 4, 2025
75ce643
Update build_wheels.yml
van-dai-vuong Jun 4, 2025
edd7f92
Update pyproject.toml
van-dai-vuong Jun 4, 2025
976ff52
Update pyproject.toml
van-dai-vuong Jun 4, 2025
ceadd10
Update pyproject.toml
van-dai-vuong Jun 4, 2025
a508478
Update pyproject.toml
van-dai-vuong Jun 4, 2025
e492060
Update build_wheels.yml
van-dai-vuong Jun 4, 2025
83ebbd7
Update build_wheels.yml
van-dai-vuong Jun 4, 2025
3e49ab3
unittest on Mac
van-dai-vuong Jun 4, 2025
ab8a398
docs workflow on macos
van-dai-vuong Jun 4, 2025
74463d2
update
van-dai-vuong Jun 4, 2025
802a208
unit test save
van-dai-vuong Jun 4, 2025
5dc81b8
Update README.md
van-dai-vuong Jun 4, 2025
9161c23
Update README.md
van-dai-vuong Jun 4, 2025
7dcf171
Update README.md
van-dai-vuong Jun 4, 2025
91011e4
change figure & link - Doc
jamesgoulet Jun 4, 2025
7279121
Change figure size - Doc
jamesgoulet Jun 4, 2025
6f8c02f
change figure size - Doc
jamesgoulet Jun 4, 2025
5c48460
update figures - doc
jamesgoulet Jun 4, 2025
5f11b42
change fig size - doc
jamesgoulet Jun 4, 2025
3743513
Change figure size - doc
jamesgoulet Jun 4, 2025
83566e6
change figure path - readme
jamesgoulet Jun 4, 2025
3bc850f
change figure size - readme
jamesgoulet Jun 4, 2025
273bd91
typo - readme
jamesgoulet Jun 4, 2025
1fd06de
35% - readme
jamesgoulet Jun 4, 2025
07a2841
38% - readme
jamesgoulet Jun 4, 2025
db931c2
40% - readme
jamesgoulet Jun 4, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 27 additions & 15 deletions .github/workflows/build_wheels.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: Build and Publish to PyPI
on:
push:
branches:
- main
- main
tags:
- 'v*'
pull_request:
Expand All @@ -24,25 +24,16 @@ jobs:
with:
python-version: "3.10"

- name: Install system dependencies
run: |
sudo apt-get update
sudo apt-get install -y libopenmpi-dev

- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install build twine

# Install pytagi from github (TODO: because cannot install from pip)
git clone --branch v0.2.0 https://github.com/lhnguyen102/cuTAGI.git
cd cuTAGI
git submodule update --init --recursive
pip install -r requirements.txt
pip install .
cd ..
#
pip install -e .

- name: Run unit tests
run: pytest canari/test
working-directory: ${{ github.workspace }}/..

- name: Clean build environment
run: rm -rf dist/ build/ *.egg-info

Expand All @@ -57,3 +48,24 @@ jobs:
run: |
twine upload --verbose --repository pypi dist/*

unit-test:
name: Unit Tests on macOS
runs-on: macos-latest

steps:
- name: Checkout code
uses: actions/checkout@v3

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: "3.10"

- name: Install dependencies
run: |
pip install pytagi
pip install -e .

- name: Run unit tests
run: pytest canari/test
working-directory: ${{ github.workspace }}/..
13 changes: 3 additions & 10 deletions .github/workflows/deploy_docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,9 @@ on:
branches:
- main

# on:
# workflow_dispatch:

# on:
# pull_request:
# branches:
# - main

jobs:
deploy:
runs-on: ubuntu-latest
runs-on: macos-latest

steps:
- name: Checkout code
Expand All @@ -31,7 +23,8 @@ jobs:
run: |
python -m pip install --upgrade pip
pip install -r docs/requirements.txt
sudo apt-get update && sudo apt-get install -y pandoc
pip install -r requirements.txt
brew install pandoc

- name: Manually add src/ to PYTHONPATH
run: echo "PYTHONPATH=$PYTHONPATH:$(pwd)/src" >> $GITHUB_ENV
Expand Down
43 changes: 26 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,35 @@
## What is Canari?
Canari is an open-source library for online change point detection in univariate time series. It has been developed in the context of long-term online monitoring where we aim for the early detection of subtle change points in the baseline response, while limiting to a minimum the number of false alarms. Because Canari’s probabilistic change-point detection relies on 1-step ahead predictions, it is inherently capable of performing short- and long-term forecasting, as well as online data imputation; One-step ahead predictions can either rely solely on a target time series, or on explanatory time series that can be non-linearly related to the target.
Canari is an open-source library for online change point detection in univariate time series. It has been developed in the context of long-term online monitoring where we aim for the early detection of subtle change points in the baseline response, while limiting to a minimum the number of false alarms.

Because Canari’s probabilistic change-point detection relies on 1-step ahead predictions, it is inherently capable of performing short- and long-term forecasting, as well as online data imputation; One-step ahead predictions can either rely solely on a target time series, or on explanatory time series that can be non-linearly related to the target.

In addition to change-point detection and forecasting capabilities, it returns an interpretable decomposition of time series into baseline components representing the irreversible responses, recurrent pattern components caused by external effects, and residual components allowing the characterization of both the model prediction and the observation uncertainties.

## How does it work?
```{figure} ../docs/_static/Canari_SSM_LSTM.png
---
scale: 30%
align: right
---
```
The methodological core behind the canary library consists of a seamless integration between [state-space models (SSM)](http://profs.polymtl.ca/jagoulet/Site/PMLCE/CH12.html) and Bayesian neural networks. On the one hand, the Gaussian SSM theory enables modelling baseline responses and residuals, while on the other, [Tractable Approximate Gaussian Inference (TAGI)](https://github.com/lhnguyen102/cuTAGI/tree/main) also enables treating all parameters and hidden states in neural networks as Gaussians. Canari uses LSTM neural networks to model recurrent patterns as well as non-linear dependencies with respect to explanatory variables. Because both the SSM and LSTM rely on the same Gaussian conditional inference mechanism, their hidden states can be inferred analytically in a same unified probabilistic framework.
The figure on the right presents an example where the raw data in red is decomposed in a baseline that is characterized by a baseline “level” component where its rate of change is described by the “trend”. The recurrent pattern is modelled by a LSTM Bayesian neural network where the training and validation set consists only in 4 years of data. The residual characterizing the model errors is itself modelled by a white “noise” component. The change point detection can be performed either online or offline after the training and validation period; The presence of change points is indicated by the probability of regime switches that rise toward 1 on several occasions.
```{figure} ../docs/_static/Canari_example.png
```

<img src="./docs/_static/Canari_SSM_LSTM.png" alt="Canari SSM LSTM" width="23%" align="right">

The methodological core behind the canary library consists of a seamless integration between [state-space models (SSM)](http://profs.polymtl.ca/jagoulet/Site/PMLCE/CH12.html) and Bayesian neural networks. On the one hand, the Gaussian SSM theory enables modelling baseline responses and residuals, while on the other, [Tractable Approximate Gaussian Inference (TAGI)](https://github.com/lhnguyen102/cuTAGI/tree/main) also enables treating all parameters and hidden states in neural networks as Gaussians.

Canari uses LSTM neural networks to model recurrent patterns as well as non-linear dependencies with respect to explanatory variables. Because both the [SSM and LSTM rely on the same Gaussian conditional](https://www.sciencedirect.com/science/article/pii/S0169207024000335) inference mechanism, their hidden states can be inferred analytically in a same unified probabilistic framework.

<img src="./docs/_static/Canari_example.png" alt="Canari example" width="40%" align="right">

The figure above presents an example where the raw data in red is decomposed in a baseline that is characterized by a baseline “level” component where its rate of change is described by the “trend”. The recurrent pattern is modelled by a LSTM Bayesian neural network where the training and validation set consists only in 4 years of data. The residual characterizing the model errors is itself modelled by a white “noise” component. The change point detection can be performed either online or offline after the training and validation period; The presence of change points is indicated by the probability of regime switches that rise toward 1 on several occasions.

## Getting started
You can get started with Canari by going through our [installation guide](https://bayes-works.github.io/canari/installation_guide.html) and [tutorials](https://bayes-works.github.io/canari/tutorials.html) covering all the main features of the library.
You can get started with Canari by going through our [tutorials](https://bayes-works.github.io/canari/tutorials.html) covering all the main features of the library.

## Installation
Canari available on PyPI, for installation, execute the following command in Terminal:
```
pip install pycanari
```
For more info. on how to install, visit our [installation guide](https://bayes-works.github.io/canari/installation_guide.html).
## Contributors
The principal developer of canary is Van Dai Vuong with the mentoring of Luong Ha Nguyen and oversight by James-A. Goulet. The major contributors to the library are
- Zhanwen Xin (Online AR, Bounded AR & several+++ bug fixes through PRs)
- David Wardan (SLSTM component & several+++ bug fixes through PRs)
The principal developer of canary is Van Dai Vuong with the mentoring of Luong Ha Nguyen and oversight by James-A. Goulet. The major contributors to the library are:
- Zhanwen Xin (Online AR, Bounded AR & several+++ bug fixes through PRs)
- David Wardan (SLSTM component & several+++ bug fixes through PRs)

# Acknowledgements
We acknowledge the financial support from Hydro-Québec and the Natural Sciences and Engineering Research Council of Canada in the development of the Canari library.
Expand All @@ -39,8 +48,8 @@ Canari is released under the MIT license.
* [Anomaly Detection with the Switching Kalman Filter for Structural Health Monitoring](https://profs.polymtl.ca/jagoulet/Site/Papers/2017_Nguyen_and_Goulet_AD-SKF.pdf) (Luong Ha Nguyen and James-A. Goulet, 2018)
* [Bayesian dynamic linear models for structural health monitoring](https://profs.polymtl.ca/jagoulet/Site/Papers/Goulet_BDLM_SHM_2017_preprint.pdf)(James-A. Goulet, 2017)
### Theses
[Analytically Tractable Bayesian Recurrent Neural Networks with Structural Health Monitoring Applications](https://profs.polymtl.ca/jagoulet/Site/Papers/DV_Thesis_2024.pdf) (Van-Dai Vuong, 2024)
*[Analytically Tractable Bayesian Recurrent Neural Networks with Structural Health Monitoring Applications](https://profs.polymtl.ca/jagoulet/Site/Papers/DV_Thesis_2024.pdf) (Van-Dai Vuong, 2024)
* [Analytical Bayesian Parameter Inference for Probabilistic Models with Engineering Applications](https://profs.polymtl.ca/jagoulet/Site/Papers/BhargobDekaThesis.pdf) (Bhargob Deka, 2022)
* [Real-time Anomaly Detection in the Behaviour of Structures](https://profs.polymtl.ca/jagoulet/Site/Papers/LHNguyen_these_2019.pdf) (Luong Ha Nguyen, 2019)
### Book chapter
* [Probabilistic machine learning for civil engineers (chapter 12 - SSM)](http://profs.polymtl.ca/jagoulet/Site/PMLCE/CH12.html) (James-A. Goulet. 2020)
* [Probabilistic machine learning for civil engineers (chapter 12 - SSM)](http://profs.polymtl.ca/jagoulet/Site/PMLCE/CH12.html) (James-A. Goulet. 2020)
Binary file modified docs/_static/Canari_SSM_LSTM.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/_static/Canari_example.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 1 addition & 3 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
project = "canari"
copyright = "2025, Van-Dai Vuong, Luong-Ha Nguyen, James-A. Goulet"
author = "Van-Dai Vuong, Luong-Ha Nguyen, James-A. Goulet"
release = "v.0.0.2"
release = "v.0.0.3"

extensions = [
"sphinx.ext.autodoc",
Expand All @@ -21,8 +21,6 @@
]
nbsphinx_execute = "never"

autodoc_mock_imports = ["pytagi"]

templates_path = ["_templates"]
exclude_patterns = []

Expand Down
48 changes: 25 additions & 23 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,19 +4,19 @@ Canari's documentation
What is Canari?
---------------

Canari is an open-source library for online change point detection in univariate time series.
It has been developed in the context of long-term online monitoring where we aim for the early
detection of subtle change points in the baseline response, while limiting to a minimum the
Canari is an open-source library for online change point detection in univariate time series.
It has been developed in the context of long-term online monitoring where we aim for the early
detection of subtle change points in the baseline response, while limiting to a minimum the
number of false alarms.

Because Canari's probabilistic change-point detection relies on 1-step ahead predictions,
it is inherently capable of performing short- and long-term forecasting, as well as online
data imputation. One-step ahead predictions can either rely solely on a target time series,
Because Canari's probabilistic change-point detection relies on 1-step ahead predictions,
it is inherently capable of performing short- and long-term forecasting, as well as online
data imputation. One-step ahead predictions can either rely solely on a target time series,
or on explanatory time series that can be non-linearly related to the target.

In addition to change-point detection and forecasting capabilities, it returns an interpretable
decomposition of time series into baseline components representing the irreversible responses,
recurrent pattern components caused by external effects, and residual components allowing the
In addition to change-point detection and forecasting capabilities, it returns an interpretable
decomposition of time series into baseline components representing the irreversible responses,
recurrent pattern components caused by external effects, and residual components allowing the
characterization of both the model prediction and the observation uncertainties.

How does it work?
Expand All @@ -26,27 +26,29 @@ How does it work?
:scale: 30%
:align: right

The methodological core behind the Canari library consists of a seamless integration between
`state-space models (SSM) <http://profs.polymtl.ca/jagoulet/Site/PMLCE/CH12.html>`_ and Bayesian
neural networks. On the one hand, the Gaussian SSM theory enables modelling baseline responses and
residuals, while on the other, `Tractable Approximate Gaussian Inference (TAGI) <https://github.com/lhnguyen102/cuTAGI/tree/main>`_
The methodological core behind the Canari library consists of a seamless integration between
`state-space models (SSM) <http://profs.polymtl.ca/jagoulet/Site/PMLCE/CH12.html>`_ and Bayesian
neural networks. On the one hand, the Gaussian SSM theory enables modelling baseline responses and
residuals, while on the other, `Tractable Approximate Gaussian Inference (TAGI) <https://github.com/lhnguyen102/cuTAGI/tree/main>`_
enables treating all parameters and hidden states in neural networks as Gaussians.

Canari uses LSTM neural networks to model recurrent patterns as well as non-linear dependencies
with respect to explanatory variables. Because both the SSM and LSTM rely on the same Gaussian
conditional inference mechanism, their hidden states can be inferred analytically in a unified
Canari uses LSTM neural networks to model recurrent patterns as well as non-linear dependencies
with respect to explanatory variables. Because both the `SSM and LSTM rely on the same Gaussian
conditional <https://www.sciencedirect.com/science/article/pii/S0169207024000335>`_ inference mechanism, their hidden states can be inferred analytically in a unified
probabilistic framework.

.. figure:: _static/Canari_example.png
:scale: 24%
:align: right

The figure above presents an example where the raw data in red is decomposed into a baseline that
is characterized by a baseline “level” component where its rate of change is described by the “trend.”
The recurrent pattern is modelled by an LSTM Bayesian neural network where the training and validation
set consists only of 4 years of data. The residual characterizing the model errors is itself modelled
The figure on the right presents an example where the raw data in red is decomposed into a baseline that
is characterized by a baseline “level” component where its rate of change is described by the “trend.”
The recurrent pattern is modelled by an LSTM Bayesian neural network where the training and validation
set consists only of 4 years of data. The residual characterizing the model errors is itself modelled
by a white “noise” component.

Change point detection can be performed either online or offline after the training and validation
period. The presence of change points is indicated by the probability of regime switches that rise
Change point detection can be performed either online or offline after the training and validation
period. The presence of change points is indicated by the probability of regime switches that rise
toward 1 on several occasions.

Getting started
Expand All @@ -69,7 +71,7 @@ The major contributors to the library are:
Acknowledgements
----------------

We acknowledge the financial support from Hydro-Québec and the Natural Sciences and
We acknowledge the financial support from Hydro-Québec and the Natural Sciences and
Engineering Research Council of Canada in the development of the Canari library.

License
Expand Down
4 changes: 0 additions & 4 deletions docs/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
Sphinx>=6.1
sphinx-book-theme>=0.3.4
nbsphinx>=0.8.8
matplotlib>=3.5
numpy>=1.22
pandas
ray[tune]
14 changes: 11 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,13 @@ build-backend = "setuptools.build_meta"

[project]
name = "pycanari"
version = "0.0.2"
description = "Open-source library for probabilistic anomaly detection in time series"
readme = "README.md"
version = "0.0.3"
description = """
Canari is an open-source library for online change point detection in univariate time series.
It has been developed in the context of long-term online monitoring where we aim for the early
detection of subtle change points in the baseline response, while limiting to a minimum the number
of false alarms.
"""
license = { text = "MIT" }
requires-python = ">=3.10"

Expand Down Expand Up @@ -39,4 +43,8 @@ package-dir = { "" = "src" }
[tool.setuptools.packages.find]
where = ["src"]

[project.urls]
Homepage = "https://bayes-works.github.io/canari/index.html"
Repository = "https://github.com/Bayes-Works/canari"


2 changes: 1 addition & 1 deletion test/saved_metric/test_model_forecast_metric.csv
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
mse
0.002291753
2.1171463e-06
File renamed without changes.