Skip to content

Add Cardington recipes #2128

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Simon Osborne (mo-sro) wants to merge 15 commits into
base: main
Choose a base branch
from
Open

Add Cardington recipes #2128

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
9242b0b
Add EIGHT Cardington recipe yaml files
mo-sro May 12, 2026
18c51ff
Change wind calculation operator to vector wind from list
mo-sro May 15, 2026
54462e2
Change to correct operator name wind.calculate_wind_vector
mo-sro May 15, 2026
aaeab54
Merge branch 'main' into cardington_recipes
mo-sro Jun 3, 2026
a5ddc0f
Merge branch 'main' into cardington_recipes
mo-sro Jun 10, 2026
bef83ea
Update Cardington variable names in YAML configuration
mo-sro Jun 12, 2026
7874d2c
Merge branch 'main' into cardington_recipes
mo-sro Jun 16, 2026
ed10559
Merge branch 'main' into cardington_recipes
mo-sro Jul 6, 2026
737416a
Update Cardington latent and sensible heat recipes
mo-sro Jul 6, 2026
5252560
Merge branch 'main' into cardington_recipes
mo-sro Jul 10, 2026
04d8c65
Modify wind gust recipe to better constrain load
mo-sro Jul 15, 2026
eab9104
Merge branch 'main' into cardington_recipes
mo-sro Jul 15, 2026
aa81619
Update all recipes for load constraints
mo-sro Jul 15, 2026
92e6e6c
Merge branch 'cardington_recipes' of github.com:/MetOffice/CSET into …
mo-sro Jul 15, 2026
3ab4d3e
Merge branch 'main' into cardington_recipes
mo-sro Aug 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions ...ET/recipes/surface_fields/cardington/cardington_latent_heat_single_point_time_series.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
category: Time series of latent heat flux at Cardington single point

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
category: Time series of latent heat flux at Cardington single point
category: Time series at Cardington

Suggest category should be something relatively general that lots of outputs might sit in (e.g. generic_spatial_plots outputs all have category "Surface Spatial Plot".
The main aim of category is to offer some organisation/cataloguing of output plots.

Suggest you have same category wording for all Cardington time series recipes.

title: "Time series of $PLOTNAME at lat_pt:52.10438, lon_pt:-0.42286 specific to Cardington gridpoint."
description: "Plots a time series of $PLOTNAME at a selected Cardington gridpoint."

steps:
- operator: read.read_cubes
file_paths: $INPUT_PATHS
model_names:
- $CARDINGTON_30_LABEL
- $MODEL_NAME

- operator: filters.filter_multiple_cubes

Cardington:
operator: constraints.combine_constraints
varname_constraint:
operator: constraints.generate_var_constraint
varname: $VARNAME
cell_methods_constraint:
operator: constraints.generate_cell_methods_constraint
cell_methods: []
varname: $VARNAME
height_constraint:
operator: constraints.generate_attribute_constraint
attribute: 'height'
value: $HEIGHT

$MODEL_NAME:
operator: constraints.combine_constraints
varname_constraint:
operator: constraints.generate_var_constraint
varname: $UM_VARNAME
cell_methods_constraint:
operator: constraints.generate_cell_methods_constraint
cell_methods: []
varname: $UM_VARNAME
pressure_level_constraint:
operator: constraints.generate_level_constraint
levels: [ ]
coordinate: "pressure"
stash_constraint:
operator: constraints.generate_attribute_constraint
attribute: STASH

- operator: misc.latent_heat_units

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding in call to filter fill_values (see example comment below) here for consistency with SH flux recipe.

- operator: misc.remove_attribute
attribute: [ 'time_coverage_end', 'time_coverage_start', 'valid_min', 'valid_max', 'date', 'date_created' ]

- operator: regrid.regrid_to_single_point
lat_pt: 52.10438
lon_pt: -0.42286
latlon_in_type: "realworld"
method: "Nearest"
boundary_margin: 0

# Make a single NetCDF with all the data inside it.
- operator: write.write_cube_to_nc
overwrite: True

# Plot the data.
- operator: plot.plot_line_series
45 changes: 45 additions & 0 deletions .../recipes/surface_fields/cardington/cardington_sensible_heat_single_point_time_series.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
category: Time series of sensible heat flux at Cardington single point

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See detailed comments in latent_heat recipe for style/naming changes, to be applied consistently in relevant places to this recipe also.

title: "Time series of $PLOTNAME at lat_pt:52.10438, lon_pt:-0.42286 specific to Cardington gridpoint."
description: "Plots a time series of $PLOTNAME at a selected Cardington gridpoint."
steps:
- operator: read.read_cubes
file_paths: $INPUT_PATHS
model_names:
- $CARDINGTON_30_LABEL
- $MODEL_NAME

- operator: misc.mask_fill_values

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After some trial/error, I was able to utilise existing CSET operator functionality to avoid introducing specific new code here as follows:

  - operator: filters.apply_mask
    mask:
       operator: filters.generate_mask
       condition: 'lt'
       value: 1.0e+10

This makes recipe slightly longer, but re-uses existing code. The logic is somewhat 'simple' as here just saying to preserve everything where data < 1e10 rather than setting specific values to NaN based on tolerances etc.

Would that approach be sufficient for most recipes (with some knowledge of 'gotcha' numbers).

My sense is the mask_fill_values is neat code, and maybe useful capability, but looking for simple implementation and avoiding too much new code where possible.


- operator: misc.sensible_heat_units
HEIGHT: $HEIGHT
CARDINGTON_VARNAMES: $CARDINGTON_VARNAMES

- operator: filters.filter_multiple_cubes
Cardington:
operator: constraints.generate_var_constraint
varname: surface_upward_sensible_heat_flux_cardington

$MODEL_NAME:
operator: constraints.generate_var_constraint
varname: surface_upward_sensible_heat_flux

- operator: misc.remove_attribute
attribute:
- time_coverage_end
- time_coverage_start
- valid_min
- valid_max
- date
- date_created

- operator: regrid.regrid_to_single_point
lat_pt: 52.10438
lon_pt: -0.42286
latlon_in_type: "realworld"
method: "Nearest"
boundary_margin: 0

- operator: write.write_cube_to_nc
overwrite: True

- operator: plot.plot_line_series
63 changes: 63 additions & 0 deletions src/CSET/recipes/surface_fields/cardington/cardington_surface_pressure_time_series.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
category: Time series of pressure at Cardington single point

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See detailed comments in previous recipes for areas to address on general approach, to be reflected in updates to this recipe also.

title: "Time series of $PLOTNAME at lat_pt:52.10438, lon_pt:-0.42286 specific to Cardington gridpoint."
description: "Plots a time series of $PLOTNAME at a selected Cardington gridpoint."

steps:
- operator: read.read_cubes
file_paths: $INPUT_PATHS
model_names:
- $CARDINGTON_05_LABEL
- $CARDINGTON_30_LABEL
- $MODEL_NAME

- operator: filters.filter_multiple_cubes

Cardington:
operator: constraints.combine_constraints
varname_constraint:
operator: constraints.generate_var_constraint
varname: $VARNAME
cell_methods_constraint:
operator: constraints.generate_cell_methods_constraint
cell_methods: []
varname: $VARNAME


$MODEL_NAME:
operator: constraints.combine_constraints
varname_constraint:
operator: constraints.generate_var_constraint
varname: $UM_VARNAME
cell_methods_constraint:
operator: constraints.generate_cell_methods_constraint
cell_methods: []
varname: $UM_VARNAME
pressure_level_constraint:
operator: constraints.generate_level_constraint
levels: []
coordinate: "pressure"
stash_constraint:
operator: constraints.generate_attribute_constraint
attribute: STASH

- operator: misc.remove_attribute
attribute: [ 'time_coverage_end', 'time_coverage_start', 'valid_min', 'valid_max', 'date', 'date_created' ]

- operator: misc.convert_units
units: 'hPa'

- operator: regrid.regrid_to_single_point
lat_pt: 52.10438
lon_pt: -0.42286
latlon_in_type: "realworld"
method: "Nearest"
boundary_margin: 0

- operator: misc.mask_fill_values

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comment in sensible_heat recipe on proposal to adopt existing apply_mask and generate_mask to satisfy this requirement.

Reminder of comment to then apply consistently across Cardington recipes.


# Make a single NetCDF with all the data inside it.
- operator: write.write_cube_to_nc
overwrite: True

# Plot the data.
- operator: plot.plot_line_series
69 changes: 69 additions & 0 deletions src/CSET/recipes/surface_fields/cardington/cardington_surface_temperature_time_series.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
category: Time series of surface temperature at Cardington single point

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See other comments for general style suggestions/requirements to be adopted consistently in this recipe.

title: "Time series of $VARNAME at lat_pt:52.10438, lon_pt:-0.42286 specific to Cardington gridpoint."
description: "Plots a time series of $VARNAME at a selected Cardington gridpoint."
steps:
# ------------------------------------------------------------

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would remove, or simplify the docstrings from recipe file itself.

As above, avoid explicit reference to "UM" for example.

Suggest the call to read.read_cubes does not require specific docstrings in final version, so would remove.

# 1. Read data
# IMPORTANT:
# - Constrain the UM *at read time* so Iris only ever loads
# a single surface_temperature cube.
# ------------------------------------------------------------

- operator: read.read_cubes
file_paths: $INPUT_PATHS
model_names:
- $CARDINGTON_05_LABEL
- $CARDINGTON_30_LABEL
- $MODEL_NAME
constraints:
$MODEL_NAME:
operator: constraints.combine_constraints
stash_constraint:
operator: constraints.generate_attribute_constraint
attribute: STASH
values:
- $UM_VARNAME
cell_methods_constraint:
operator: constraints.generate_cell_methods_constraint
cell_methods: [ 'maximum' ]

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this recipe be updated to reference Tmax therefore?

Are there intended to be equivalents for Tmin, Tmean etc?

Could the required cell_method potentially be user-defined variable $CELL_METHOD if appetite/requirement to cover all options?

coord: 'time'
interval: '1 hour'
preprocessors:

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Preprocessors is new to me.

Is it useful to adopt similar preprocessor step in previous recipes to remove unwanted attributes also?

This would seem to be neater approach than a 'random' call to these operators lower down in recipe.

Aim to adopt consistent approach where possible across the recipe set.

Alternatively, why are scalar_coords removed in this recipe, but seemingly not in other recipes? Is this still needed?

- operator: misc.remove_scalar_coords
coords:
- forecast_reference_time
- realization

# IMPORTANT: Global temperature-only prune.

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this still valid/required if constrain by varname now done in read.read_cubes step?

# filter_multiple_cubes must have exactly ONE key here,
# otherwise non-temperature cubes will survive and break convert_units.
- operator: filters.filter_multiple_cubes
temperature:
operator: constraints.generate_var_constraint
varname: $VARNAME

- operator: misc.remove_attribute
attribute:
- time_coverage_end
- time_coverage_start
- valid_min
- valid_max
- date
- date_created

- operator: misc.convert_units
units: 'Celsius'

- operator: regrid.regrid_to_single_point
lat_pt: 52.10438
lon_pt: -0.42286
latlon_in_type: "realworld"
method: "Nearest"
boundary_margin: 0

- operator: misc.mask_fill_values

- operator: write.write_cube_to_nc
overwrite: True

- operator: plot.plot_line_series
51 changes: 33 additions & 18 deletions ...temperature_single_point_time_series.yaml → ...temperature_single_point_time_series.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,40 +1,53 @@
category: Time series of air temperature at Cardington single point
title: Time series of 'air_temperature' at lat_pt:52.10438, lon_pt:-0.42286 specific to Cardington gridpoint.
description: Plots a time series of the air temperature at a selected Cardington gridpoint.
title: "Time series of $PLOTNAME at lat_pt:52.10438, lon_pt:-0.42286 specific to Cardington gridpoint."
description: "Plots a time series of $PLOTNAME at a selected Cardington gridpoint."

steps:
- operator: read.read_cubes
file_paths: $INPUT_PATHS
model_names: ['Cardington','$MODEL_NAME']
constraint:
model_names:
- $CARDINGTON_05_LABEL
- $CARDINGTON_30_LABEL
- $MODEL_NAME

- operator: filters.filter_multiple_cubes

Cardington:
operator: constraints.combine_constraints
varname_constraint:
operator: constraints.generate_var_constraint
varname: 'air_temperature'
varname: $VARNAME
cell_methods_constraint:
operator: constraints.generate_cell_methods_constraint
cell_methods: []
varname: 'air_temperature'
varname: $VARNAME
height_constraint:
operator: constraints.generate_attribute_constraint
attribute: 'height'
value: $HEIGHT

$MODEL_NAME:
operator: constraints.combine_constraints
varname_constraint:
operator: constraints.generate_var_constraint
varname: $UM_VARNAME
cell_methods_constraint:
operator: constraints.generate_cell_methods_constraint
cell_methods: []
varname: $UM_VARNAME
pressure_level_constraint:
operator: constraints.generate_level_constraint
levels: []
coordinate: "pressure"

- operator: misc.remove_attribute
attribute: ['time_coverage_end', 'time_coverage_start', 'valid_min', 'valid_max', 'date', 'date_created']

- operator: filters.filter_multiple_cubes
cardington:
stash_constraint:
operator: constraints.generate_attribute_constraint
attribute: 'height'
value: 2.0
model:
operator: constraints.generate_attribute_constraint
# This currently only works for UM. Maybe we need another operator to make this nicer?
attribute: STASH

- operator: misc.remove_attribute
attribute: [ 'time_coverage_end', 'time_coverage_start', 'valid_min', 'valid_max', 'date', 'date_created' ]

- operator: misc.convert_units
units: 'Celsius'
units: 'celsius'

- operator: regrid.regrid_to_single_point
lat_pt: 52.10438
Expand All @@ -43,6 +56,8 @@ steps:
method: "Nearest"
boundary_margin: 0

- operator: misc.mask_fill_values

# Make a single NetCDF with all the data inside it.
- operator: write.write_cube_to_nc
overwrite: True
Expand Down
52 changes: 52 additions & 0 deletions ...SET/recipes/surface_fields/cardington/cardington_visibility_single_point_time_series.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
category: Time series of visibility at Cardington single point
title: "Time series of $PLOTNAME at lat_pt:52.10438, lon_pt:-0.42286 specific to Cardington gridpoint."
description: "Plots a time series of $PLOTNAME at a selected Cardington gridpoint."

steps:
- operator: read.read_cubes
file_paths: $INPUT_PATHS
model_names:
- $CARDINGTON_05_LABEL
- $CARDINGTON_30_LABEL
- $MODEL_NAME

- operator: filters.filter_multiple_cubes
Cardington:
operator: constraints.generate_var_constraint
varname: $VARNAME

$MODEL_NAME:
operator: constraints.combine_constraints
varname_constraint:
operator: constraints.generate_var_constraint
varname: $UM_VARNAME
pressure_level_constraint:
operator: constraints.generate_level_constraint
levels: []
coordinate: "pressure"
stash_constraint:
operator: constraints.generate_attribute_constraint
attribute: STASH

- operator: misc.remove_attribute
attribute:
- time_coverage_end
- time_coverage_start
- valid_min
- valid_max
- date
- date_created

- operator: misc.convert_visibility_to_km

@ukmo-huw-lewis ukmo-huw-lewis Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check if this required, or if
read._convert_cube_units_callback() in read.py doesn't already cover this? And if not, why not?

That function includes test of varnames for "visibility" and then converts units to km.


- operator: regrid.regrid_to_single_point
lat_pt: 52.10438
lon_pt: -0.42286
latlon_in_type: "realworld"
method: "Nearest"
boundary_margin: 0

- operator: write.write_cube_to_nc
overwrite: True

- operator: plot.plot_line_series
Loading