Skip to content

Scores [Categorical] Probability of False Detection - #2425

Merged
James Warner (jwarner8) merged 24 commits into
mainfrom
2404_pfd
Aug 26, 2026
Merged

Scores [Categorical] Probability of False Detection#2425
James Warner (jwarner8) merged 24 commits into
mainfrom
2404_pfd

Conversation

@jwarner8

@jwarner8 James Warner (jwarner8) commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Addresses #2404

Includes refactor of categorical scores operators, and adds support for false detection and frequency bias, along with corresponding tests.

Microsoft CoPilot (web chat) was used to help draft the unit tests.

Contribution checklist

Aim to have all relevant checks ticked off before merging. See the developer's guide for more detail.

  • Documentation has been updated to reflect change.
  • New code has tests, and affected old tests have been updated.
  • All tests and CI checks pass.
  • Ensured the pull request title is descriptive.
  • Attributed any Generative AI, such as GitHub Copilot, used in this PR.
  • Marked the PR as ready to review.

@jwarner8 James Warner (jwarner8) added the enhancement New feature or request label Aug 24, 2026
@github-actions

github-actions Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor
Total coverage: 93% (HTML report)
Name                                                              Stmts   Miss Branch BrPart  Cover
---------------------------------------------------------------------------------------------------
src/CSET/__init__.py                                                105      0     14      0   100%
src/CSET/_common.py                                                 156      0     54      0   100%
src/CSET/cset_workflow/app/fetch_fcst/bin/fetch_data.py             117     28     26      0    78%
src/CSET/cset_workflow/app/fetch_nimrod/bin/fetch_nimrod.py          81      8     28     11    83%
src/CSET/cset_workflow/app/finish_website/bin/finish_website.py      79      2      8      2    95%
src/CSET/cset_workflow/app/parbake_recipes/bin/parbake.py            29      0      8      0   100%
src/CSET/cset_workflow/app/send_email/bin/send_email.py              25      0      4      0   100%
src/CSET/cset_workflow/lib/python/jinja_utils.py                     17      0      6      0   100%
src/CSET/extract_workflow.py                                        103      1     26      1    98%
src/CSET/graph.py                                                    44      0     14      0   100%
src/CSET/operators/__init__.py                                       89      0     26      0   100%
src/CSET/operators/_atmospheric_constants.py                          9      0      0      0   100%
src/CSET/operators/_colormaps.py                                    249      3     72      4    98%
src/CSET/operators/_stash_to_lfric.py                                 3      0      0      0   100%
src/CSET/operators/_utils.py                                        190      8     72      6    95%
src/CSET/operators/ageofair.py                                      142      7     64      5    94%
src/CSET/operators/aggregate.py                                      77      1     22      1    98%
src/CSET/operators/aviation.py                                       61      0     18      0   100%
src/CSET/operators/collapse.py                                      155      8     72      3    93%
src/CSET/operators/constraints.py                                   113      7     50      3    93%
src/CSET/operators/convection.py                                     38      4     10      2    88%
src/CSET/operators/ensembles.py                                      27      0     14      0   100%
src/CSET/operators/feature.py                                        44      0     10      0   100%
src/CSET/operators/filters.py                                        67      2     30      0    98%
src/CSET/operators/fluxes.py                                         41      0     10      0   100%
src/CSET/operators/humidity.py                                      135      0     52      0   100%
src/CSET/operators/imageprocessing.py                                57      0     16      0   100%
src/CSET/operators/mesoscale.py                                      18      0      2      0   100%
src/CSET/operators/misc.py                                          172      1     72      3    98%
src/CSET/operators/plot.py                                         1120    157    412     71    83%
src/CSET/operators/power_spectrum.py                                 98      3     30      3    95%
src/CSET/operators/precipitation.py                                 204      2     92      2    99%
src/CSET/operators/pressure.py                                       41      0     12      0   100%
src/CSET/operators/read.py                                          439     22    190     17    94%
src/CSET/operators/regrid.py                                        147      1     70      3    98%
src/CSET/operators/scoreswrappers.py                                217     22     66      5    88%
src/CSET/operators/temperature.py                                   121      0     32      0   100%
src/CSET/operators/transect.py                                       63      0     24      0   100%
src/CSET/operators/wind.py                                           46      3     10      2    91%
src/CSET/operators/write.py                                          15      0      6      0   100%
src/CSET/recipes/__init__.py                                        104      0     28      0   100%
src/CSET/sample_data/__init__.py                                      0      0      0      0   100%
---------------------------------------------------------------------------------------------------
TOTAL                                                              5058    290   1772    144    93%

@jwarner8
James Warner (jwarner8) marked this pull request as ready for review August 25, 2026 15:26

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implementation looks good and well structured. Need to verify typehinting and return types though.

Comment thread src/CSET/operators/scoreswrappers.py Outdated
Comment thread src/CSET/operators/scoreswrappers.py Outdated

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, to to update the references then happy for you to merge.

Comment thread src/CSET/recipes/verification/surface_categorical_model_obs_pfd.yaml Outdated
Comment thread src/CSET/recipes/verification/surface_categorical_model_obs_pfd.yaml Outdated
…d.yaml

Co-authored-by: James Ross <james.ross@metoffice.gov.uk>
@jwarner8
James Warner (jwarner8) merged commit 77a00b9 into main Aug 26, 2026
9 checks passed
@jwarner8
James Warner (jwarner8) deleted the 2404_pfd branch August 26, 2026 09:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants