Skip to content

Add ESACCI biomass benchmark (2015-2024) and observation-error test - #4064

Merged
mdietze merged 1 commit into
PecanProject:developfrom
Tejas7007:GH-ilamb-esacci
Aug 18, 2026
Merged

Add ESACCI biomass benchmark (2015-2024) and observation-error test#4064
mdietze merged 1 commit into
PecanProject:developfrom
Tejas7007:GH-ilamb-esacci

Conversation

@Tejas7007

Copy link
Copy Markdown
Contributor

Adds the ESACCI Biomass v7.0 product as a second, more recent (2015-2024) aboveground-biomass benchmark with a per-pixel uncertainty layer. Includes a converter to ILAMB-compatible netCDF (regridded to the XuSaatchi grid, dry biomass converted to carbon with the IPCC 0.47 fraction), a scoring and observation-error module that adds observation error to the ensemble spread, tests, and figures. The ensemble is equally overconfident against XuSaatchi, ESACCI 2020, and ESACCI 2024, and the overconfidence persists after accounting for observation error.

Description

Extends the biomass benchmarking in inst/ilamb/ to use ESA CCI Biomass v7.0 as a second, independent above-ground biomass benchmark alongside the existing XuSaatchi product, and adds an observation-error test to the ensemble calibration assessment.

Contents:

  • convert_esacci_to_ilamb.py converts the ESACCI aggregated GeoTIFF to an ILAMB-compatible netCDF on the XuSaatchi grid. It extracts the requested years, regrids 0.25 to 0.5 degree, and applies the IPCC woody carbon fraction (0.47) to convert dry biomass to carbon density, since XuSaatchi and the model output are in carbon. It writes a biomass variable and a biomass_sd per-pixel uncertainty variable. The geospatial imports are lazy so the constants can be imported without the geospatial stack installed.

  • score_esacci.py samples the downscaled ensemble at the assimilation sites and computes the calibration diagnostics against ESACCI and XuSaatchi, then adds observation error to the ensemble spread and re-tests. Its coverage_with_obs_error and spread_skill_with_obs_error functions are general and apply to any benchmark that carries an uncertainty estimate.

  • esacci_figures.py produces three figures: the ESACCI-versus-XuSaatchi comparison, the calibration against all three benchmarks, and the observation-error result.

  • test_esacci.py covers the year-to-band mapping, the carbon conversion, and the observation-error diagnostics, using only NumPy and the calibration module.

  • README_esacci.md documents the data source (CEDA), the units conversion, and how to run everything.

Results at the assimilation sites: the ensemble is equally overconfident against XuSaatchi (~2010), ESACCI 2020, and ESACCI 2024 (spread-skill ratio ~0.07 in all three), so the finding is not specific to an older benchmark. Accounting for observation error does not rescue the calibration: even when the observation error is estimated generously from the difference between the XuSaatchi and ESACCI products, coverage rises only from ~0.02 to ~0.10 and the ratio barely moves, both far below their calibrated targets.

This builds on the ensemble calibration diagnostics already in inst/ilamb/ and imports ensemble_calibration.py from that addition, so the per-region and per-benchmark numbers are computed consistently with the domain-wide ones.

Motivation and Context

The existing above-ground biomass benchmark (XuSaatchi) is frozen near 2010 and carries no per-pixel uncertainty, while the reanalysis runs through 2024. ESACCI provides annual maps spanning the reanalysis period with an uncertainty layer, which makes it possible to (1) check whether the ensemble overconfidence is specific to the older benchmark by scoring against contemporary 2020 and 2024 observations, and (2) propagate observation error into the calibration to test whether the overconfidence survives when the observation is not treated as perfect. Both checks are addressed here.

Review Time Estimate

  • Immediately
  • Within one week
  • When possible

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My change requires a change to the documentation.
  • My name is in the list of CITATION.cff
  • I agree that PEcAn Project may distribute my contribution under any or all of
    • the same license as the existing code,
    • and/or the BSD 3-clause license.
  • I have updated the CHANGELOG.md.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

Adds the ESA CCI Biomass v7.0 product as a second, more recent (2015-2024)
aboveground-biomass benchmark with a per-pixel uncertainty layer. Includes a
converter to ILAMB-compatible netCDF (regridded to the XuSaatchi grid, dry
biomass converted to carbon with the IPCC 0.47 fraction), a scoring and
observation-error module that adds observation error to the ensemble spread, a
notebook, tests, and figures. The ensemble is equally overconfident against
XuSaatchi, ESACCI 2020, and ESACCI 2024, and the overconfidence persists after
accounting for observation error.

@ankurdesai ankurdesai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@mdietze
mdietze added this pull request to the merge queue Aug 18, 2026
Merged via the queue into PecanProject:develop with commit 45e31d5 Aug 18, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants