Skip to content

Add datacube variables support as MLM input and output dimensions - #94

Merged
fmigneault merged 25 commits into
mainfrom
data-cube-variables
Jul 17, 2025
Merged

Add datacube variables support as MLM input and output dimensions#94
fmigneault merged 25 commits into
mainfrom
data-cube-variables

Conversation

@fmigneault

Copy link
Copy Markdown
Collaborator

Description

  • Add variables properties to Model Input Object to allow specifying the relevant data variables used by the model, with cross-references to the datacube extension (relates to #90).
  • Add bands and variables properties to Model Output Object to allow specifying the relevant bands or variables produced by the model if any applies.
  • Add downscaling to Tasks as common operation for climate variable models.
  • Refactor the JSON schema to check for bands and variables references within both mlm:input and mlm:output. If either location detects that either bands or variables is provided, their corresponding sets of extensions providing relevant descriptions are verified.
  • Refactor the JSON schema mlm:output property to employ a ModelOutput object definition rather than directly provided properties nested under the array.
  • Refactor the JSON schema to allow the omission of bands under mlm:input if the variables property is provided.
  • Fix missing encoding="utf-8" parameters in open calls leading to failing parsing of example JSON STAC Item when they contain non-ASCII characters.
  • Fix parsing of PYTHON_PATH in Makefile for Windows OS.

Related Issue

Type of Change

  • 📚 Examples, docs, tutorials or dependencies update;
  • 🔧 Bug fix (non-breaking change which fixes an issue);
  • 🥂 Improvement (non-breaking change which improves an existing feature);
  • 🚀 New feature (non-breaking change which adds functionality);
  • 💥 Breaking change (fix or feature that would cause existing functionality to change);
  • 🔐 Security fix.

Checklist

  • I've read the CONTRIBUTING.md guide;
  • I've updated the code style using make check;
  • I've written tests for all new methods and classes that I created;
  • I've written the docstring in Google format for all the methods and classes that I used.

@fmigneault fmigneault self-assigned this Jul 9, 2025
@fmigneault fmigneault changed the title Data cube variables Add datacube variables support as MLM input and output dimensions Jul 9, 2025
@fmigneault
fmigneault requested review from m-mohr and rbavery July 9, 2025 05:23
@fmigneault

fmigneault commented Jul 9, 2025

Copy link
Copy Markdown
Collaborator Author

@huard @Nazim-crim FYI

Initial MLM definition using datacube variables for describing a climate/ML model.
I have found a ERA5 downscaling model on GitHub code that should resemble the Ouranos UNet.
I used that reference for creating an example in the meantime. Only difference should be the actual variables definition using the RDPS/HRDPS instead of ERA5.

Comment thread README.md Outdated
@rbavery

rbavery commented Jul 9, 2025

Copy link
Copy Markdown
Collaborator

Refactor the JSON schema to allow the omission of bands under mlm:input if the variables property is provided.

Is the reverse allowed, specifying bands but omitting variables ? I think the README might need to be updated, it currently says both are REQUIRED

@fmigneault
fmigneault marked this pull request as ready for review July 10, 2025 03:48
@fmigneault
fmigneault requested a review from rbavery July 10, 2025 03:48
@fmigneault

Copy link
Copy Markdown
Collaborator Author

@rbavery All comments should be addressed. Let me know if there is something else to adjust.

@fmigneault fmigneault mentioned this pull request Jul 15, 2025
10 tasks
rbavery
rbavery previously approved these changes Jul 17, 2025
@fmigneault
fmigneault merged commit 9452941 into main Jul 17, 2025
8 checks passed
@fmigneault
fmigneault deleted the data-cube-variables branch July 17, 2025 14:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support CF-variables as alternate I/O references

2 participants