`s2swa_32bit_oneapi` & `atmaero_oneapi` compile failures on GaeaC6 w/spack-stack v2.1.0

[gspetro-NOAA]

Description

On Gaea-C6, all RT configurations compile with spack-stack v2.1.0 except:

• s2swa_32bit_oneapi: GOCART2G_MieMod.F90(227): #error: number of arguments doesn't match.
• atmaero_oneapi: GOCART2G_MieMod.F90(227): #error: number of arguments doesn't match.

To Reproduce:

1. Run the s2swa_32bit_oneapi or atmaero_oneapicompile task (or any tests that use them) along with the spack-stack v2.1.0 stack used in PR Upgrade WM to spack-stack 2.1.x #3174 .
2. Tests will fail.

Additional context

I'm not sure who should own the troubleshooting for this issue, but likely the developers whose tests run these compiles.

Output

Latest run_dir on GaeaC6: /gpfs/f6/epic/proj-shared/Gillian.Petro/RT_RUNDIRS/Gillian.Petro/FV3_RT/rt_2388594
Old run_dir on GaeaC6: /gpfs/f6/epic/proj-shared/Gillian.Petro/RT_RUNDIRS/Gillian.Petro/FV3_RT/rt_3838122

[gspetro-NOAA]

@lipan-NOAA @bbakernoaa Do you know who might be able to address this issue?

[gspetro-NOAA]

After testing some potential fixes w/@benkozi, we can resolve the GOCART2G_MieMod.F90(227): #error: number of arguments doesn't match. error on Gaea C6. However, the fixes we applied to GOCART2G_MieMod.F90 likely need to be applied to a number of GOCART files, since fixing GOCART2G_MieMod.F90 now brings up a similar error in a different file: Chem_AeroGeneric.F90(72): #error: number of arguments doesn't match. It's unclear how many files might require this fix. @benkozi has proposed 3 potential options:

• Apply a patch during the build to the problem files on C6.
• Patch GOCART directly. This seems challenging and would take longer since the GOCART submod is hosted in a NASA repository. However, this fix is the most robust.
• Adjust something with the oneapi compiler I'm not aware of.

It seems like a GOCART patch would be preferable, but we're open to ideas!

The fix applied to GOCART2G_MieMod.F90 was:

diff --git a/Process_Library/GOCART2G_MieMod.F90 b/Process_Library/GOCART2G_MieMod.F90
index 2ed4ef7..a76f741 100644
--- a/Process_Library/GOCART2G_MieMod.F90
+++ b/Process_Library/GOCART2G_MieMod.F90
@@ -159,7 +159,7 @@ CONTAINS
      integer :: status
 
 #define NF_VERIFY_(expr) rc = expr; if (rc /= 0) return
-#define __NF_STAT__ stat=status); NF_VERIFY_(status
+#define __NF_STAT__ stat=status
 
      rc = 0
      this%table_name = MieFile
@@ -224,24 +224,24 @@ CONTAINS
 !     Get the table contents
 !     -------------------------------------
 
-      allocate(channels_table(nch_table), __NF_STAT__)
-      allocate(rh_table(nrh_table), __NF_STAT__)
-      allocate(reff_table(nrh_table,nbin_table), __NF_STAT__)
-      allocate(bext_table(nch_table,nrh_table,nbin_table), __NF_STAT__)
-      allocate(bsca_table(nch_table,nrh_table,nbin_table), __NF_STAT__)
-      allocate(bbck_table(nch_table,nrh_table,nbin_table),  __NF_STAT__)
+      allocate(channels_table(nch_table), __NF_STAT__); NF_VERIFY_(status)
+      allocate(rh_table(nrh_table), __NF_STAT__); NF_VERIFY_(status)
+      allocate(reff_table(nrh_table,nbin_table), __NF_STAT__); NF_VERIFY_(status)
+      allocate(bext_table(nch_table,nrh_table,nbin_table), __NF_STAT__); NF_VERIFY_(status)
+      allocate(bsca_table(nch_table,nrh_table,nbin_table), __NF_STAT__); NF_VERIFY_(status)
+      allocate(bbck_table(nch_table,nrh_table,nbin_table),  __NF_STAT__); NF_VERIFY_(status)
       allocate(g_table(nch_table,nrh_table,nbin_table), stat = rc )
-      allocate(pback_table(nch_table,nrh_table,nbin_table,nPol_table),  __NF_STAT__)
-      allocate(gf_table(nrh_table,nbin_table), __NF_STAT__)
-      allocate(rhop_table(nrh_table,nbin_table), __NF_STAT__)
-      allocate(rhod_table(nrh_table,nbin_table), __NF_STAT__)
-      allocate(vol_table(nrh_table,nbin_table), __NF_STAT__)
-      allocate(area_table(nrh_table,nbin_table), __NF_STAT__)
-      allocate(refr_table(nch_table,nrh_table,nbin_table), __NF_STAT__)
-      allocate(refi_table(nch_table,nrh_table,nbin_table), __NF_STAT__)
+      allocate(pback_table(nch_table,nrh_table,nbin_table,nPol_table),  __NF_STAT__); NF_VERIFY_(status)
+      allocate(gf_table(nrh_table,nbin_table), __NF_STAT__); NF_VERIFY_(status)
+      allocate(rhop_table(nrh_table,nbin_table), __NF_STAT__); NF_VERIFY_(status)
+      allocate(rhod_table(nrh_table,nbin_table), __NF_STAT__); NF_VERIFY_(status)
+      allocate(vol_table(nrh_table,nbin_table), __NF_STAT__); NF_VERIFY_(status)
+      allocate(area_table(nrh_table,nbin_table), __NF_STAT__); NF_VERIFY_(status)
+      allocate(refr_table(nch_table,nrh_table,nbin_table), __NF_STAT__); NF_VERIFY_(status)
+      allocate(refi_table(nch_table,nrh_table,nbin_table), __NF_STAT__); NF_VERIFY_(status)
 
       if ( nmom_ > 0 ) then
-         allocate(pmom_table(nch_table,nrh_table,nbin_table,nmom_table,nPol_table), __NF_STAT__)
+         allocate(pmom_table(nch_table,nrh_table,nbin_table,nmom_table,nPol_table), __NF_STAT__); NF_VERIFY_(status)
       end if
       NF_VERIFY_(nf90_inq_varid(ncid,'lambda',ivarid))
       NF_VERIFY_(nf90_get_var(ncid,ivarid,channels_table))
@@ -340,26 +340,26 @@ CONTAINS
       this%nMom = nmom_
       this%nPol = nPol_table
 
-      allocate (this%rh(this%nrh), __NF_STAT__)
-      allocate (this%reff(this%nrh,this%nbin), __NF_STAT__)
-      allocate (this%bext(this%nrh,this%nch,this%nbin), __NF_STAT__)
-      allocate (this%bsca(this%nrh,this%nch,this%nbin), __NF_STAT__)
-      allocate (this%bbck(this%nrh,this%nch,this%nbin), __NF_STAT__)
-      allocate (this%g(this%nrh,this%nch,this%nbin),    __NF_STAT__)
-      allocate (pback(this%nrh,this%nch,this%nbin,this%nPol),    __NF_STAT__)
+      allocate (this%rh(this%nrh), __NF_STAT__); NF_VERIFY_(status)
+      allocate (this%reff(this%nrh,this%nbin), __NF_STAT__); NF_VERIFY_(status)
+      allocate (this%bext(this%nrh,this%nch,this%nbin), __NF_STAT__); NF_VERIFY_(status)
+      allocate (this%bsca(this%nrh,this%nch,this%nbin), __NF_STAT__); NF_VERIFY_(status)
+      allocate (this%bbck(this%nrh,this%nch,this%nbin), __NF_STAT__); NF_VERIFY_(status)
+      allocate (this%g(this%nrh,this%nch,this%nbin),    __NF_STAT__); NF_VERIFY_(status)
+      allocate (pback(this%nrh,this%nch,this%nbin,this%nPol),    __NF_STAT__); NF_VERIFY_(status)
       if ( nmom_ > 0 ) then
-         allocate (this%pmom(this%nrh,this%nch,this%nbin,this%nMom,this%nPol),    __NF_STAT__)
+         allocate (this%pmom(this%nrh,this%nch,this%nbin,this%nMom,this%nPol),    __NF_STAT__); NF_VERIFY_(status)
       end if
-      allocate (this%gf(this%nrh,this%nbin),    __NF_STAT__)
-      allocate (this%rhop(this%nrh,this%nbin),    __NF_STAT__)
-      allocate (this%rhod(this%nrh,this%nbin),    __NF_STAT__)
-      allocate (this%vol(this%nrh,this%nbin),    __NF_STAT__)
-      allocate (this%area(this%nrh,this%nbin),    __NF_STAT__)
-      allocate (this%refr(this%nrh,this%nch,this%nbin), __NF_STAT__)
-      allocate (this%refi(this%nrh,this%nch,this%nbin), __NF_STAT__)
-      allocate (this%wavelengths(this%nch), __NF_STAT__)
-      allocate (this%p11(this%nrh,this%nch,this%nbin),    __NF_STAT__)
-      allocate (this%p22(this%nrh,this%nch,this%nbin),    __NF_STAT__)
+      allocate (this%gf(this%nrh,this%nbin),    __NF_STAT__); NF_VERIFY_(status)
+      allocate (this%rhop(this%nrh,this%nbin),    __NF_STAT__); NF_VERIFY_(status)
+      allocate (this%rhod(this%nrh,this%nbin),    __NF_STAT__); NF_VERIFY_(status)
+      allocate (this%vol(this%nrh,this%nbin),    __NF_STAT__); NF_VERIFY_(status)
+      allocate (this%area(this%nrh,this%nbin),    __NF_STAT__); NF_VERIFY_(status)
+      allocate (this%refr(this%nrh,this%nch,this%nbin), __NF_STAT__); NF_VERIFY_(status)
+      allocate (this%refi(this%nrh,this%nch,this%nbin), __NF_STAT__); NF_VERIFY_(status)
+      allocate (this%wavelengths(this%nch), __NF_STAT__); NF_VERIFY_(status)
+      allocate (this%p11(this%nrh,this%nch,this%nbin),    __NF_STAT__); NF_VERIFY_(status)
+      allocate (this%p22(this%nrh,this%nch,this%nbin),    __NF_STAT__); NF_VERIFY_(status)
 
 !     Preserve the full RH structure of the input table
       this%rh = rh_table 

[DusanJovic-NOAA]

It looks like the issue is a bug in the Fortran preprocessor (fpp) in the current compiler version used on Gaea C6.

Since the problem is not in the code, maybe the simplest solution is to temporarily patch the CMakeLists.txt, for example like this:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index fd3c535a..05957ee5 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -235,6 +235,33 @@ endif()
 ###############################################################################
 if(UFS_GOCART)
   add_subdirectory(GOCART)
+  #
+  # This ugly hack is necessary to avoid the error from buggy fpp in Intel compiler version 2025.2.1
+  #
+  #   If:
+  # 1) we are on Gaea C6 AND
+  # 2) the Fortran compiler is IntelLLVM version 2025.2.1 AND
+  # 3) an fpp from an earlier compiler version exists THEN
+  #   explicitly set the name of the fpp preprocessor executable for all targets in GOCART
+  #
+  # Once newer Intel compiler (with fixed fpp) is installed on Gaea C6 this should be removed
+  #
+  if(CMAKE_Platform STREQUAL "gaeac6.oneapi" AND
+     CMAKE_Fortran_COMPILER_ID STREQUAL "IntelLLVM" AND CMAKE_Fortran_COMPILER_VERSION VERSION_EQUAL 2025.2.1 AND
+     EXISTS "/opt/intel/oneapi/compiler/2025.0/bin/fpp")
+
+    target_compile_options(Process_Library   PRIVATE -fpp-name=/opt/intel/oneapi/compiler/2025.0/bin/fpp -Qoption,fpp,-P)
+    target_compile_options(Chem_Shared2G     PRIVATE -fpp-name=/opt/intel/oneapi/compiler/2025.0/bin/fpp -Qoption,fpp,-P)
+    target_compile_options(GA_Environment    PRIVATE -fpp-name=/opt/intel/oneapi/compiler/2025.0/bin/fpp -Qoption,fpp,-P)
+    target_compile_options(CA2G_GridComp     PRIVATE -fpp-name=/opt/intel/oneapi/compiler/2025.0/bin/fpp -Qoption,fpp,-P)
+    target_compile_options(DU2G_GridComp     PRIVATE -fpp-name=/opt/intel/oneapi/compiler/2025.0/bin/fpp -Qoption,fpp,-P)
+    target_compile_options(NI2G_GridComp     PRIVATE -fpp-name=/opt/intel/oneapi/compiler/2025.0/bin/fpp -Qoption,fpp,-P)
+    target_compile_options(SS2G_GridComp     PRIVATE -fpp-name=/opt/intel/oneapi/compiler/2025.0/bin/fpp -Qoption,fpp,-P)
+    target_compile_options(SU2G_GridComp     PRIVATE -fpp-name=/opt/intel/oneapi/compiler/2025.0/bin/fpp -Qoption,fpp,-P)
+    target_compile_options(GOCART2G_GridComp PRIVATE -fpp-name=/opt/intel/oneapi/compiler/2025.0/bin/fpp -Qoption,fpp,-P)
+    target_compile_options(Aerosol_GridComp  PRIVATE -fpp-name=/opt/intel/oneapi/compiler/2025.0/bin/fpp -Qoption,fpp,-P)
+    target_compile_options(UFS_Aerosols      PRIVATE -fpp-name=/opt/intel/oneapi/compiler/2025.0/bin/fpp -Qoption,fpp,-P)
+  endif()
 endif()

 ###############################################################################```

[gspetro-NOAA]

It looks like the issue is a bug in the Fortran preprocessor (fpp) in the current compiler version used on Gaea C6.

Since the problem is not in the code, maybe the simplest solution is to temporarily patch the CMakeLists.txt.

Thanks! Your solution seems to have worked. The compile passes, and the test fails with the divide by zero error that we're working on fixing for other tests! 🙂