Merge branch 'develop' into automated-reporting-and-visualization

Author: ayushman1210

CHANGELOG.md

@@ -9,6 +9,7 @@ For more information about this file see also [Keep a Changelog](http://keepacha
 ## Unreleased
 
 ### Added
+- Added ensemble calibration diagnostics to `inst/ilamb/` in PEcAn.benchmark (rank histogram, spread-skill ratio, coverage, reliability) for assessing whether an ensemble's spread is well calibrated against observations, complementing the ensemble-mean benchmarking.
 - Added `make_scorecard.sh` and documentation to `inst/ilamb/` in PEcAn.benchmark for generating and serving the ILAMB HTML scorecard.
 - New function `PEcAn.utils::netcdf2df()` flattens all dims and vars of a netCDF into a dataframe,
     with units attached as an attribute.

base/utils/data/standard_vars.csv

@@ -0,0 +1,43 @@
+# setEnsemblePaths leaves all path components other than siteid
+# identical across sites. This is an issue for our dataset, because
+# each site has a different location and date range.
+# To use site-specific grid id, we need to string-replace each siteid
+
+#' Set grid cell names (not exported)
+#'
+#' @param s settings object
+#'
+#' @returns updated settings with met paths including grid cell
+#' @export
+#'
+#' @examples
+id2grid <- function(s) {
+  # replacing in place to preserve names
+  for (p in seq_along(s$run$inputs$met$path)) {
+    s$run$inputs$met$path[[p]] <- gsub(
+      pattern = s$run$site$id,
+      replacement = s$run$site$ERA5_grid_cell,
+      x = s$run$inputs$met$path[[p]]
+    )
+  }
+  s
+}
+
+#' Set start and end dates (not exported)
+#'
+#' @param s settings object
+#'
+#' @returns updated settings object with met path including correct years
+#'
+#' @examples
+dates2grid <- function(s) {
+  for (p in seq_along(s$run$inputs$met$path)) {
+    s$run$inputs$met$path[[p]] <- gsub(
+      pattern = "DATES-HERE",
+      replacement = paste0(s$run$site$met.start, ".",
+                           s$run$site$met.end),
+      x = s$run$inputs$met$path[[p]]
+    )
+  }
+  s
+}

models/peprmt/R/helpers.R

@@ -2,8 +2,6 @@
 title: "Running PEPRMT Using PEcAn"
 author: 
   - "Abby Lewis"
-  - "Aritra Dey"
-  - "David LeBauer"
 format:
   html: default
   pdf: default
@@ -14,74 +12,71 @@ fig-height: 6
 fig-dpi: 300
 ---
 
-# Introduction {#introduction}
+# Introduction
 
 Welcome to this PEcAn workflow notebook! This notebook will guide you through running the PEPRMT model using PEcAn's programmatic interface.
 
 ## What Is PEcAn?
 
 PEcAn (Predictive Ecosystem Analyzer) is a scientific workflow system designed to make ecosystem modeling more transparent, repeatable, and accessible. It helps researchers:
 
--   Run ecosystem models with standardized inputs and outputs
--   Perform uncertainty analysis on model parameters
--   Compare model predictions with observations
--   Share and reproduce scientific workflows
+- Run ecosystem models with standardized inputs and outputs
+- Perform uncertainty analysis on model parameters
+- Compare model predictions with observations
+- Share and reproduce scientific workflows
 
 ## What This Notebook Does
 
-This notebook demonstrates how to:
+This notebook is patterned after [PEcAn demo 1](https://pecanproject.github.io/pecan-documentation/develop/run-demo-1.html), which was written by Aritra Dey and David LeBauer. It demonstrates how to:
 
 1.  Set up and configure a PEcAn workflow
-2.  Run an ecosystem model simulation
+2.  Run an ecosystem model simulation using the PEPRMT model
 3.  Analyze and visualize the results
 
 ### The Scenario Being Modeled:
 
-We are modeling greenhouse gas dynamics (gross primary productivity, ecosystem respiration, and methane emissions) at five sites. The model configuration uses the PEPRMT-Tidal process-based ecosystem model (Oikawa et al. 2024). 
+We are modeling greenhouse gas dynamics (gross primary productivity, ecosystem respiration, and methane emissions) at five sites. The model configuration uses the PEPRMT-Tidal process-based ecosystem model (Oikawa et al. 2024).
 
 The simulation is run for the duration of data at each site, which ranges from 2011 to 2021.
 
-This scenario is designed to be a minimal, reproducible example to demonstrate how to run PEPRMT within the PEcAn workflow. In later steps, this same framework can be extended to include more ensemble members, longer time periods, or alternative meteorological inputs.
+This scenario is designed to be a minimal, reproducible example to demonstrate how to run PEPRMT within the PEcAn workflow. In later steps, this same framework can be extended to include more ensemble members, longer time periods, or alternative meteorological inputs (e.g., see `pecan/models/peprmt/demo_run/02_input_demo.qmd` and `pecan/models/peprmt/demo_run/03_event_demo.qmd`).
 
 ## Prerequisites
 
 Before running this notebook, make sure you have:
 
--   All the PEcAn packages installed. You can install all PEcAn packages and their dependencies by running the following command in the root of your PEcAn repository. This chunk does not run automatically since it will overwrite a version of PEcAn you installed by other methods. Run it by hand when needed.
+- All the PEcAn packages installed. You can install all PEcAn packages and their dependencies by running the following command in the root of your PEcAn repository. This chunk does not run automatically since it will overwrite a version of PEcAn you installed by other methods. Run it by hand when needed.
 
-```{r, eval = FALSE}
+```{r packages, eval = FALSE}
 
 # Enable repository from pecanproject
 options(repos = c(
   pecanproject = 'https://pecanproject.r-universe.dev',
   CRAN = 'https://cloud.r-project.org'))
 # Download and install PEcAn.all in R
 install.packages(c('PEcAn.all', 'PEcAn.PEPRMT'))
-```
-
--   Alternatively, install from GitHub.
 
-```{r, eval = FALSE}         
+# Alternatively, install from GitHub
 remotes::install_github("pecanproject/pecan", subdir = "base/all", 
                         ref = "develop", force = T)
 remotes::install_github("pecanproject/pecan", subdir = "models/peprmt", 
                         ref = "develop")
 ```
 
-
-```{r}
-#Set working directory
-here::i_am("models/peprmt/demo_run/run_pecan_peprmt.qmd")
-```
-
-- A valid `pecan.xml` configuration file or use the example provided: `pecan/models/peprmt/demo_run/settings.xml`
+- You will also need a valid `pecan.xml` configuration file, such as the example provided: `pecan/models/peprmt/demo_run/settings.xml`
 
 ## How to Use This Notebook
 
-1.  Each section is clearly marked with a heading
+1.  Each section is marked with a heading
 2.  Code chunks are provided with explanations
 3.  You can run the code chunks sequentially
 4.  Once you have successfully run the demo, you can modify parameters to configure new runs and analyses
+5.  By default, the demo runs within `pecan/models/peprmt/demo_run/`.
+
+```{r wd}
+#Set working directory
+here::i_am("models/peprmt/demo_run/01_run_pecan_peprmt.qmd")
+```
 
 **Objective:**
 
@@ -103,8 +98,14 @@ First, we need to load the PEcAn R packages. These packages provide all the func
 # Load the PEcAn.all package, which includes all necessary PEcAn functionality
 library("PEcAn.all")
 library("PEcAn.PEPRMT")
-set.seed(20260326)
+
+# Set "seed" to make outputs reproducible
+set.seed(20260326) 
+
+# prints logger messages into the notebook instead of to the terminal
 logger.setUseConsole(console = TRUE, stderr = FALSE)
+# prints messages with levels INFO or higher (WARNING, ERROR, SEVERE)
+# For even more detail, change to "DEBUG"
 logger.setLevel("INFO")
 ```
 
@@ -126,8 +127,8 @@ settings_path <- here::here("models/peprmt/demo_run/settings_inputdemo.xml")
 
 After specifying the path to the `pecan.xml` file, the next step involves reading and preparing these settings. PEcAn provides utilities to process and validate the configurations before execution begins.
 
--   `PEcAn.settings::read.settings(settings_path)`: Reads the `pecan.xml` file and converts it to an R list object.
--   `PEcAn.settings::prepare.settings(settings)`: Prepares and validates settings. It sets defaults for missing fields, changes file paths to absolute paths, and generally ensures consistency.
+- `PEcAn.settings::read.settings(settings_path)`: Reads the `pecan.xml` file and converts it to an R list object.
+- `PEcAn.settings::prepare.settings(settings)`: Prepares and validates settings. It sets defaults for missing fields, changes file paths to absolute paths, and generally ensures consistency.
 
 ```{r read-prepare-settings}
 # Read the settings from the pecan.xml file
@@ -141,7 +142,7 @@ settings <- PEcAn.settings::prepare.settings(settings)
 
 Once the settings have been read and prepared, it is useful to inspect the structure of the `settings` object. This object is an R list containing all parameters and configurations for the PEcAn workflow.
 
--   `str(settings)` displays the internal structure of the `settings` object. This shows how the settings are represented in R and is useful for debugging and verifying settings.
+- `str(settings)` displays the internal structure of the `settings` object. This shows how the settings are represented in R and is useful for debugging and verifying settings.
 
 ```{r explore-settings}
 str(settings)
@@ -165,22 +166,25 @@ Editing the more interesting settings to change the PFT (`settings$pfts`) or ext
 
 The directory structure created by PEcAn for this demo run will look like this:
 
-```
+```         
 demo_outdir/               # Root output directory
+├── ensemble.samples.<ensemble_id>.Rdata # contains the selected parameter values for each ensemble member
 ├── run/                   # Configuration & execution metadata
 │   ├── runs.txt           # List of run IDs (one per model realization)
-│   ├── <runid>/           # Model-specific config copies (sometimes)
-│   └── config.*           # Generated model configs (e.g., SIPNET)
+│   └── <runid>/           # Model-specific configs, including run job and driver data
 ├── out/                   # Raw model outputs by run ID
-│   └── <runid>/           # E.g., daily PEPRMT output files
+│   └── <runid>/           # E.g., PEPRMT output files
+├── STATUS                 # timestamp and ok/error status of each workflow step
+├── pecan.*.xml            # copies of the run settings saved at different steps
+├── run_manifest.csv       # info about site, PFT, etc for each runid
+└── samples.Rdata          # draws from parameter distributions
+
 ```
 
 The root output directory is defined here as `demo_outdir/` by `settings$outdir`. This directory contains log and record files from the PEcAn workflow. They provide a detailed record of how data was generated and are key components of the analysis metadata and provenance. These can be useful for debugging as well as for downstream analysis.
 
 Key subdirectories include `run/` and `out/` that contain files used to configure and run the model, files generated by the underlying ecosystem model, and PEcAn standard outputs used in downstream analyses. These are described in subsequent sections.
 
-Additional outputs include logs, a `STATUS` file that records the steps of the workflow along with timestamps and whether each step was successful, and a copy of the `pecan.*.xml` file.
-
 # Write Model Configuration Files
 
 This step generates the model-specific configuration files and scripts that will be used to run the ecosystem model. The process involves generating PEPRMT configuration files using the `runModule.run.write.configs()` function.
@@ -193,27 +197,25 @@ settings <- PEcAn.workflow::runModule.run.write.configs(settings)
 
 This section executes the actual model simulations and retrieves the results. The process is managed by PEcAn's workflow system, which handles the execution of your chosen ecosystem model.
 
--   `runModule_start_model_runs(settings)`: This function initiates the model runs based on your configuration. It manages the execution of your chosen ecosystem model, using the configuration files generated in the previous step.
+- `runModule_start_model_runs(settings)`: This function initiates the model runs based on your configuration. It manages the execution of your chosen ecosystem model, using the configuration files generated in the previous step.
 
 ```{r run-model}
 PEcAn.workflow::runModule_start_model_runs(settings)
 ```
 
-
 This step generates raw model outputs in model-specific format (in this case, `out.csv`) as well as log files.
 
 # Extract Model Results and Prepare for Analysis
 
 After the model simulation completes, we need to extract the results and prepare them for analysis. This involves:
 
-1. Reading the run ID
-2. Setting up output paths
-3. Defining time period
-4. Loading model output
-5. Convert to a standard format
+1.  Reading the run ID
+2.  Setting up output paths
+3.  Defining time period
+4.  Loading model output
+5.  Convert to a standard format
 
-Here we read output one site at a time to account for differing simulation periods.
-For runs where all sites have the same dates, it should also work to pass the entire runid vector to a single `read.output` call.
+Here we read output one site at a time to account for differing simulation periods. For runs where all sites have the same dates, it should also work to pass the entire runid vector to a single `read.output` call.
 
 ```{r get-plot-vars}
 runid <- read.csv(file.path(settings$outdir, "runs_manifest.csv"))$run_id
@@ -236,26 +238,19 @@ model_output <- purrr::map2_dfr(
 available_vars <- names(model_output)[!names(model_output) %in% c("posix", "time_bounds")]
 ```
 
-Running this code will convert model specific output files into a standardized netCDF ([year].nc) that can be downloaded for visualization and analysis (R, Matlab, ncview, panoply, etc). This is a key step, because this standardization enables PEcAn to apply downstream analyses to outputs from different ecosystem models.
+Running this code will convert model specific output files into a standardized netCDF (\[year\].nc) that can be downloaded for visualization and analysis (R, Matlab, ncview, panoply, etc). This is a key step, because this standardization enables PEcAn to apply downstream analyses to outputs from different ecosystem models.
 
 # Display Available Model Variables
 
-This section shows all the variables that are available in the model output. These variables represent different ecosystem processes and states that the model has simulated.
+This section shows all the variables that are available in the model output. These variables represent different ecosystem processes and states that the model has simulated. Here we are referencing PEcAn standard variables for documentation.
 
 ```{r show-variables, echo=FALSE}
 vars_df <- PEcAn.utils::standard_vars |>
   dplyr::select(
     Variable = Variable.Name,
     Description = Long.name
   ) |>
-  dplyr::filter(Variable %in% available_vars) |>
-  # TODO: add year to PEcAn.utils::standard vars
-  dplyr::bind_rows(
-    dplyr::tibble(
-      Variable = "year",
-      Description = "Year"
-    )
-  )
+  dplyr::filter(Variable %in% available_vars)
 
 vars_df$Description[is.na(vars_df$Description)] <- "(No description available)"
 knitr::kable(vars_df, caption = "Model Output Variables and Descriptions")
@@ -328,6 +323,7 @@ If you want to remove all files and directories created by this workflow and sta
 ```
 
 # Session Info
+
 This section prints your R session information for reproducibility.
 
 ```{r session-info}