update for dev2 realtime run by haiqinli · Pull Request #1498 · NOAA-EMC/rrfs-workflow

haiqinli · 2026-05-18T15:56:46Z

DESCRIPTION OF CHANGES:

Update the workflow to run with 4 aerosol tracers: smoke-fine, dust-fine, dust-coarse, and unspc-fine (anthropogenic emission);
Use regrid submodule;
Use namelist option of "config_mp_aero_emission" to turn on/off aerosol emission for TEMPO MP.

TESTS CONDUCTED:

CONUS12km test run: /scratch4/BMC/acomp/Haiqin.Li/OPSROOT/smoke_12km

Machines/Platforms:

WCOSS2
- Cactus/Dogwood
- Acorn
RDHPCS
- Hera
- Ursa
- Orion
- Hercules

Test cases:

ISSUE:

Fixes the issue(s) mentioned in #9999

CONTRIBUTORS (optional):

Jordan Schnell

 INTERP_WEIGHTS_DIR=${CHEM_INPUT}/grids/interpolation_weights/
+SCRIP_FILES_DIR=${CHEM_INPUT}/grids/scrip_files/
+# Now set the same for the scrip file:
+if [[ "${SCRIP_FILES_DIR}/mpas_${MESH_NAME}_scrip.nc" ]]; then


+GOES_INPUT=/scratch4/BMC/zrtrr/jdduda/smoke_mask/GOES
+# output directories
+GOES_OUTPUTDIR=${DATA}
+OUTPUTFILE=${UMBRELLA_PREP_CHEM_DATA}/goes.aod.init.nc


 export CHEM_GROUPS="smoke" # any combinations of "smoke, anthro, pollen, dust, rwc"
-export EBB_DCYCLE=1
+export EBB_DCYCLE=-1
+export REGRID_WRAPPER_DIR="${HOMErrfs}/sorc/regrid-wrapper"


+   ncwa -O -a LAY,COL "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"
+   ncwa -O -a LAY,COL,TSTEP "${NEMO_STACKFILE_PROCESSED}" "${NEMO_STACKFILE_PROCESSED}"
+   # Cast the stack parameters through time
+   ncap2 -O -s 'e_ant_pt_in_unspc_fine[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_unspc_fine' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"


+   ncwa -O -a LAY,COL,TSTEP "${NEMO_STACKFILE_PROCESSED}" "${NEMO_STACKFILE_PROCESSED}"
+   # Cast the stack parameters through time
+   ncap2 -O -s 'e_ant_pt_in_unspc_fine[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_unspc_fine' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"
+   ncap2 -O -s 'e_ant_pt_in_unspc_coarse[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_unspc_coarse' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"


guoqing-noaa · 2026-05-18T16:00:43Z

@@ -0,0 +1,19 @@
+prepend_path("MODULEPATH", "/gpfs/f6/bil-fire8/world-shared/Benjamin.Koziol/mpas-aerosols/spack-stack/envs/mpas-aerosols/modules/Core")


We usually don't do this way. Let's discuss this first.

guoqing-noaa · 2026-05-18T16:03:35Z

    config_gfl_sub3d = 1/" namelist.atmosphere
 fi

+if [[ "${MESH_NAME^^}" == "FWX1.25km" ]]; then


We don't use if switches based on MESH_NAME inside j-jobs and ex-scripts.

Also config_lbc_w = 'zero' should be a standard option now, right?

guoqing-noaa · 2026-05-18T16:04:33Z

    -e "${mpasout_replacement}"  "${PARMrrfs}"/streams.atmosphere  > streams.atmosphere
 #
-if [[ "${mpasout_interval,,}" == "none" ]]; then  # remove the da_state stream for coldstart only forecasts
+#if [[ "${mpasout_interval,,}" == "none" ]]; then  # remove the da_state stream for coldstart only forecasts


We cannot comment out this part. Let us talk about a better way to address the Chem needs.

I agree. When I run with rrfsx/MPAS-Model, we don't need to comment out this part. However, we have to comment out this when run with ufs-community/MPAS-Model

Fixed as suggested. Thanks!

guoqing-noaa · 2026-05-18T16:05:03Z

 # 3. CHEM_INPUT             -- location of interpolated files, ready to be used
-# 4. MESH_NAME                -- name of the MPAS domain, required to know if we have weights or data intepolated to the domain 
+# 4. MESH_NAME                -- name of the MPAS domain, required to know if we have weights or data intepolated to the domain
+# TEST COMMIT


Line 18 can be removed

Will remove line 18. Thanks!

Removed as suggested.

guoqing-noaa · 2026-05-18T16:07:45Z

We cannot use the ufs-community/MPAS-Model directly as it misses the output_timelevels and output_done_marker

I have a pending PR (ufs-community/MPAS-Model#240). Should I update the PR with output_timelevels and output_done_marker? Thanks!

We have two routes:
(1) We get ufs-community/MPAS-Model#240 merged first; then merge the changes to rrfsx/MPAS-Model which already has output_timelevels and output_done_marker
(2) If you need the above PR #240 change earlier, you can create a PR to rrfsx/MPAS-Model.

I just submitted a PR (RRFSx/MPAS-Model#25) to rrfsx/MPAS-Model. Thanks!

guoqing-noaa · 2026-05-18T16:08:42Z

 if [[ "${CHEM_GROUPS,,}" == *pollen* ]]; then
   if [[ -s "${UMBRELLA_PREP_CHEM_DATA}/bio.init.nc" ]]; then
-      sed -i "\${e} cat ${PARMrrfs}/chemistry/streams.atmosphere.pollen" streams.atmosphere # append before the last line (i.e. </stream>)
+      sed -i "\$e cat ${PARMrrfs}/chemistry/streams.atmosphere.pollen" streams.atmosphere # append before the last line (i.e. </stream>)


why change ${e} to $e?

Will change $e back to ${e}. Thanks!

for some reason the brackets cause this command to fail

I cannot remember exactly, but I think I manually tested ${e} in this command and it worked.
I will double check this. If it fails, we are fine to use $e. Thanks!

guoqing-noaa · 2026-05-18T16:10:39Z

+#   if [[ "${CHEM_GROUPS,,}" == *ssalt* ]]; then
+#     ncap2 -O -s "ssalt_fine=1.e-12*qv" init.nc init.nc
+#     ncap2 -O -s "ssalt_coarse=1.e-12*qv" init.nc init.nc
+#   fi


Lines 35-61: Are they still needed? If not, we can remove them

jordanschnell · 2026-05-20T16:33:24Z

+fi # Is GRA2PES listed as one of the anthro inventories?
+
+
+# Now for NEMO emissions


there are additional helper scripts needed for the NEMO implemetnation
chem_create_merge_dates_ann.py
chem_get_merge_date.py
chem_merge_emissions.py
chem_merge_pt_emissions.py

jordanschnell · 2026-05-20T16:36:49Z

+  source "${USHrrfs}"/chem_prep_goes_aod.sh
+fi
+
 if [[ "${CHEM_GROUP}" == "dust" ]]; then


we will need to update the FIX files for dust based on the name changes in MPAS-Model

jordanschnell · 2026-05-20T16:37:54Z

      err_exit
    fi
  fi
+  ncap2 -O -s 'frp_in=frp_in.ttl($nkwildfire)' -s 'fre_in=fre_in.ttl($nkwildfire)' "${EMISFILE}" "${EMISFILE}"


please see latest commits - this dimension summing should only occur for the actual RAVE files, the dummy files will already have the correct dimensions

guoqing-noaa · 2026-05-26T18:52:38Z

-	url = https://github.com/RRFSx/MPAS-Model
-        branch = gsl/develop
+	url = https://github.com/haiqinli/MPAS-Model
+        branch = gsl/develop-mp-emission


@haiqinli Could you merge the latest rrfs-mpas-jedi branch and then revert changes in lines 8-9? Thanks!

Updated as suggested. Thanks!

+   # Cast the stack parameters through time
+   ncap2 -O -s 'e_ant_pt_in_unspc_fine[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_unspc_fine' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"
+   ncap2 -O -s 'e_ant_pt_in_unspc_coarse[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_unspc_coarse' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"
+   ncap2 -O -s 'e_ant_pt_in_co[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_co' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"


+   ncap2 -O -s 'e_ant_pt_in_unspc_fine[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_unspc_fine' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"
+   ncap2 -O -s 'e_ant_pt_in_unspc_coarse[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_unspc_coarse' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"
+   ncap2 -O -s 'e_ant_pt_in_co[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_co' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"
+   ncap2 -O -s 'e_ant_pt_in_nox[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_nox' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"


+   ncap2 -O -s 'e_ant_pt_in_unspc_coarse[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_unspc_coarse' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"
+   ncap2 -O -s 'e_ant_pt_in_co[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_co' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"
+   ncap2 -O -s 'e_ant_pt_in_nox[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_nox' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"
+   ncap2 -O -s 'e_ant_pt_in_nh3[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_nh3' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"


+   ncap2 -O -s 'e_ant_pt_in_co[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_co' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"
+   ncap2 -O -s 'e_ant_pt_in_nox[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_nox' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"
+   ncap2 -O -s 'e_ant_pt_in_nh3[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_nh3' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"
+   ncap2 -O -s 'e_ant_pt_in_so2[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_so2' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"


+   ncap2 -O -s 'e_ant_pt_in_nox[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_nox' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"
+   ncap2 -O -s 'e_ant_pt_in_nh3[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_nh3' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"
+   ncap2 -O -s 'e_ant_pt_in_so2[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_so2' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"
+   ncap2 -O -s 'e_ant_pt_in_nh4_a_fine[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_nh4_a_fine' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"


+   ncap2 -O -s 'e_ant_pt_in_nh3[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_nh3' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"
+   ncap2 -O -s 'e_ant_pt_in_so2[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_so2' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"
+   ncap2 -O -s 'e_ant_pt_in_nh4_a_fine[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_nh4_a_fine' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"
+   ncap2 -O -s 'e_ant_pt_in_no3_a_fine[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_no3_a_fine' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"


+   ncap2 -O -s 'e_ant_pt_in_so2[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_so2' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"
+   ncap2 -O -s 'e_ant_pt_in_nh4_a_fine[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_nh4_a_fine' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"
+   ncap2 -O -s 'e_ant_pt_in_no3_a_fine[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_no3_a_fine' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"
+   ncap2 -O -s 'e_ant_pt_in_so4_a_fine[$Time,$TSTEP,$nanthro_pt]=e_ant_pt_in_so4_a_fine' "${NEMO_EMISFILE_PT_PROCESSED}" "${NEMO_EMISFILE_PT_PROCESSED}"


+    ncrename -v NOx,e_bb_in_nox "${EMISFILE2}"
    ln -sf "${EMISFILE2}" "${EMISFILE}"
+    # TODO temporary fix until YAML options are built into regriddder
+    ncap2 -O -s 'frp_in=frp_in.ttl($nkwildfire)' -s 'fre_in=fre_in.ttl($nkwildfire)' "${EMISFILE}" "${EMISFILE}"


+    ncrename -v NOx,e_bb_in_nox "${EMISFILE2}"
    ln -sf "${EMISFILE2}" "${EMISFILE}"
+    # TODO temporary fix until YAML options are built into regriddder
+    ncap2 -O -s 'frp_in=frp_in.ttl($nkwildfire)' -s 'fre_in=fre_in.ttl($nkwildfire)' "${EMISFILE}" "${EMISFILE}"


guoqing-noaa · 2026-06-02T17:22:28Z

+  export MPASSIT_DX=1000.0
+  export MPASSIT_REF_LAT=37.0
+  export MPASSIT_REF_LON=-119.0
+


Lines 100-111: remove the "garnetSm1km" section

Removed as suggested. Thanks!

guoqing-noaa · 2026-06-02T17:23:52Z

-  }
-  trap cleanup SIGINT SIGTERM
+else
+  echo "Info: Not slurm nor PBS"


merge the latest rrfs-mpas-jedi branch so that the changes on the left side are not overwritten accidentally.

Merged the latest rrfs-mpas-jedi branch.

guoqing-noaa · 2026-06-02T17:24:50Z

+#!/usr/bin/env python3
+import argparse
+import numpy as np
+


Add some documentation here about what this is.

guoqing-noaa · 2026-06-02T17:43:23Z

 	url = https://github.com/pyDAmonitor/pyDAmonitor.git
        ignore = all
+[submodule "sorc/regrid-wrapper"]
+	path = sorc/regrid-wrapper


sorc/chem-regrid-wrapper

+      # file for that same year and month (e.g., matching 201705*)
+      if [[ ! -r "${testfile}" ]]; then
+         YYYYMM="${testdate:0:6}"
+         first_available=$(ls -1 "${INDIR_NEMO}/${isect}/"emis_mole_${isect}_${YYYYMM}*.ncf 2>/dev/null | head -n 1)


+      # file for that same year and month (e.g., matching 201705*)
+      if [[ ! -r "${testfile}" ]]; then
+         YYYYMM="${testdate:0:6}"
+         first_available=$(ls -1 "${INDIR_NEMO}/${isect}/"emis_mole_${isect}_${YYYYMM}*.ncf 2>/dev/null | head -n 1)


haiqinli added 2 commits May 12, 2026 16:00

"update for chemistry"

98bae88

"update to use GRA2PES anthropogenic emission"

a46ffc1

haiqinli requested review from BenjaminBlake-NOAA, MatthewPyle-NOAA and ShunLiu-NOAA as code owners May 18, 2026 15:56

github-advanced-security AI found potential problems May 18, 2026

View reviewed changes

guoqing-noaa reviewed May 18, 2026

View reviewed changes

Merge branch 'rrfs-mpas-jedi' into rrfs-mpas-jedi-smoke

7bb7357

jordanschnell reviewed May 20, 2026

View reviewed changes

guoqing-noaa mentioned this pull request May 21, 2026

add a new option 'chemrrfs' to build.all #1505

Merged

guoqing-noaa reviewed May 26, 2026

View reviewed changes

"update MPAS-Model and latest cheMPAS-fire configurations"

c633501

github-advanced-security AI found potential problems Jun 1, 2026

View reviewed changes

guoqing-noaa reviewed Jun 2, 2026

View reviewed changes

haiqinli added 2 commits June 4, 2026 19:42

Merge branch 'rrfs-mpas-jedi' into rrfs-mpas-jedi-smoke

53935fd

"sync from cheMPAS-Fire"

e26668a

github-advanced-security AI found potential problems Jun 4, 2026

View reviewed changes

haiqinli added 2 commits June 8, 2026 16:09

"update the chemistry namelists"

916142c

"update following reviewer's suggestions"

66d9e45

Merge branch 'rrfs-mpas-jedi' into rrfs-mpas-jedi-smoke

215d540

		@@ -0,0 +1,19 @@
		prepend_path("MODULEPATH", "/gpfs/f6/bil-fire8/world-shared/Benjamin.Koziol/mpas-aerosols/spack-stack/envs/mpas-aerosols/modules/Core")

		fi # Is GRA2PES listed as one of the anthro inventories?


		# Now for NEMO emissions

Conversation

haiqinli commented May 18, 2026

DESCRIPTION OF CHANGES:

TESTS CONDUCTED:

Machines/Platforms:

Test cases:

ISSUE:

CONTRIBUTORS (optional):

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

haiqinli May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

guoqing-noaa May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

haiqinli May 19, 2026 •

edited

Loading

guoqing-noaa May 26, 2026 •

edited

Loading