Add util/check_libirc.sh to check for libirc.so used in spack-stack environments (#1639)

This PR adds a utility util/check_libirc.so that checks shared libraries and executables in a spack-stack environment for links to libirc.so. The documentation adds more details on why this is needed and when/how to use it, and I added the tool to the existing GitHub actions tests. The purpose of this utility is to catch issues with spack-stack environments only.

I also created a wiki page that will provide details on how to avoid linking to libirc.so in downstream applications: https://github.com/JCSDA/spack-stack/wiki/Intel-oneAPI-compilers-and-libirc.so

The newly added check found libirc.so in several met binaries. I fixed that and improved the existing fix for parallel-netcdf in JCSDA/spack#543.

Author: climbfuji

.github/workflows/ubuntu-ci-x86_64-oneapi-ifx.yaml

@@ -182,6 +182,9 @@ jobs:
           spack install --fail-fast --source --no-check-signature 2>&1 | tee log.install.oneifx-2024.2.0-buildcache.unified-env
           spack buildcache create -u /home/ubuntu/spack-stack/build-cache/
 
+          # Check for libirc.so in shared libraries (returns 1=error if found)
+          ${SPACK_STACK_DIR}/util/check_libirc.sh
+
           # Remove binary cache for next round of concretization
           if [ "$USE_BINARY_CACHE" = true ] ; then
             spack mirror rm local-binary
@@ -222,6 +225,9 @@ jobs:
           # Check shared libraries
           ${SPACK_STACK_DIR}/util/ldd_check.py $SPACK_ENV 2>&1 | tee log.ldd_check
 
+          # Check for libirc.so in shared libraries (returns 1=error if found)
+          ${SPACK_STACK_DIR}/util/check_libirc.sh
+
           # Create modules
           spack clean -a
           spack module tcl refresh -y

.github/workflows/ubuntu-ci-x86_64-oneapi.yaml

@@ -182,6 +182,9 @@ jobs:
           spack install --fail-fast --source --no-check-signature 2>&1 | tee log.install.oneapi-2024.2.0-buildcache.unified-env
           spack buildcache create -u /home/ubuntu/spack-stack/build-cache/
 
+          # Check for libirc.so in shared libraries (returns 1=error if found)
+          ${SPACK_STACK_DIR}/util/check_libirc.sh
+
           # Remove binary cache for next round of concretization
           if [ "$USE_BINARY_CACHE" = true ] ; then
             spack mirror rm local-binary
@@ -222,6 +225,9 @@ jobs:
           # Check shared libraries
           ${SPACK_STACK_DIR}/util/ldd_check.py $SPACK_ENV 2>&1 | tee log.ldd_check
 
+          # Check for libirc.so in shared libraries (returns 1=error if found)
+          ${SPACK_STACK_DIR}/util/check_libirc.sh
+
           # Create modules
           spack clean -a
           spack module tcl refresh -y

configs/common/packages.yaml

@@ -170,7 +170,10 @@ packages:
     require: '@2.53.0 ~shared ~f2py'
     variants: '+pflogger'
   met:
-    require: '@12.0.1 +python +grib2'
+    require:
+    - '@12.0.1'
+    - '+python'
+    - '+grib2'
   metplus:
     require: '@6.0.0'
   metis:
@@ -209,7 +212,8 @@ packages:
     require:
     - '@2.6.2'
   parallel-netcdf:
-    require: '@1.12.3'
+    require:
+    - '@1.12.3'
   pflogger:
     require: '+mpi'
   pixman:

configs/common/packages_oneapi.yaml

@@ -24,6 +24,12 @@ packages:
     require: '%gcc'
   libbsd:
     require: '%gcc'
+  met:
+    require:
+    - '+shared-intel'
+  parallel-netcdf:
+    require:
+    - '+shared-intel'
   qt:
     require:
     - '%gcc'

configs/sites/tier1/atlantis/compilers.yaml

@@ -88,8 +88,8 @@ compilers:
     - llvm/20.1.5
     environment:
       append_path:
-        LD_LIBRARY_PATH: /gpfs/neptune/spack-stack/llvm-20.1.5/lib
+        LD_LIBRARY_PATH: /gpfs/neptune/spack-stack/llvm-20.1.5/lib:/gpfs/neptune/spack-stack/llvm-20.1.5/lib/x86_64-unknown-linux-gnu
       prepend_path:
         MODULEPATH: /gpfs/neptune/spack-stack/openmpi-5.0.6/llvm-20.1.5/modulefiles:/gpfs/neptune/spack-stack/llvm-20.1.5/modulefiles
-    extra_rpaths: [/gpfs/neptune/spack-stack/llvm-20.1.5/lib]
+    extra_rpaths: [/gpfs/neptune/spack-stack/llvm-20.1.5/lib, /gpfs/neptune/spack-stack/llvm-20.1.5/lib/x86_64-unknown-linux-gnu ]
 

doc/source/Utilities.rst

@@ -24,22 +24,36 @@ In any case, the identification of any duplicates will yield a return code of 1.
 check_permissions.sh
 ------------------------------
 
-The utility located at util/check_permissions.sh can be run inside any spack-stack environment directory intended for multiple users (i.e., on an HPC or cloud platform). It will return errors if the environment directory is inaccessible to non-owning users and groups (i.e., if o+rx not set), as well as if any directories or files have permissions that make them inaccessible to other users.
+The utility located at ``util/check_permissions.sh`` can be run inside any spack-stack environment directory intended for multiple users (i.e., on an HPC or cloud platform). It will return errors if the environment directory is inaccessible to non-owning users and groups (i.e., if o+rx not set), as well as if any directories or files have permissions that make them inaccessible to other users.
 
 .. _LDD_Checker:
 
 ------------------------------
 ldd_check.py (Linux only)
 ------------------------------
 
-The util/ldd_check.py utility should be run for new installations to ensure that no shared library or executable that uses shared libraries is missing a shared library dependency. If the script returns a warning for a given file, this may indicate that Spack's RPATH substitution has not been properly applied. In some instances, missing library dependencies may not indicate a problem, such as a library that is intended to be found through $LD_LIBRARY_PATH after, say, a compiler or MPI environment module is loaded. Though these paths should probably also be RPATH-ified, such instances of harmless missing dependencies may be ignored with ldd_check.py's ``--ignore`` option by specifying a Python regular expression to be excluded from consideration (see example below), or can be permanently whitelisted by modifying the ``whitelist`` variable at the top of the ldd_check.py script itself (in which case please submit a PR). The script searches the 'install/' subdirectory of a given path and runs ``ldd`` on all shared objects. The base path to be search can be specified as a lone positional argument, and by default is the current directory. In practice, this should be ``$SPACK_ENV`` for the environment in question. This utility is available for Linux only.
+The ``util/ldd_check.py`` utility should be run for new installations to ensure that no shared library or executable that uses shared libraries is missing a shared library dependency. If the script returns a warning for a given file, this may indicate that Spack's RPATH substitution has not been properly applied. In some instances, missing library dependencies may not indicate a problem, such as a library that is intended to be found through $LD_LIBRARY_PATH after, say, a compiler or MPI environment module is loaded. Though these paths should probably also be RPATH-ified, such instances of harmless missing dependencies may be ignored with ldd_check.py's ``--ignore`` option by specifying a Python regular expression to be excluded from consideration (see example below), or can be permanently whitelisted by modifying the ``whitelist`` variable at the top of the ldd_check.py script itself (in which case please submit a PR). The script searches the 'install/' subdirectory of a given path and runs ``ldd`` on all shared objects. The base path to be search can be specified as a lone positional argument, and by default is the current directory. In practice, this should be ``$SPACK_ENV`` for the environment in question. This utility is available for Linux only.
 
 .. code-block:: console
 
-   cd $SPACK_ENV && ../../util/ldd_check.py
+   cd ${SPACK_ENV} && ../../util/ldd_check.py
    # - OR -
    util/ldd_check.py $SPACK_ENV --ignore '^libfoo.+' # check for missing shared dependencies, but ignore missing libfoo*
 
+.. _Libirc_Checker:
+
+------------------------------
+check_libirc.sh (Linux only)
+------------------------------
+
+The ``util/check_libirc.sh`` utility should be run for new installations with Intel oneAPI (``icx``, ``icpx``, ``ifort`` or ``ifx``). In an active environment after a successful ``spack install``, execute the following command to check if any of the shared libraries or executables is linked to ``libirc.so``. See https://github.com/JCSDA/spack-stack/issues/1436 for some background context and why we want need to avoid ``libirc.so``. If ``libirc.so`` is linked to a shared library or executable in a spack-stack environment, please create an issue in the spack-stack GitHub repository (https://github.com/JCSDA/spack-stack/issues). For downstream applications, see the spack-stack wiki (https://github.com/JCSDA/spack-stack/wiki/Intel-oneAPI-compilers-and-libirc.so).
+
+.. code-block:: console
+
+   cd ${SPACK_ENV} && ../../util/check_libirc.sh
+   # - OR -
+   cd ${SPACK_STACK_DIR} && ./check_libirc.sh
+
 .. _Parallel_Install:
 
 ------------------------------

spack

@@ -0,0 +1,27 @@
+#!/usr/bin/env bash
+
+ierror=0
+
+searchfor="libirc.so"
+
+echo "Checking shared libraries for ${searchfor} ..."
+for file in `find ${SPACK_ENV} -type f -iname '*.so*'`; do
+  # Return code from grep is 1 if not found, 0 if found
+  ldd $file 2>&1 | grep "${searchfor}"
+  if [[ $? -eq 0 ]]; then
+    echo "Found ${searchfor} linked in ${file}"
+    ierror=1
+  fi
+done
+
+echo "Checking executables for ${searchfor} ..."
+for file in `find ${SPACK_ENV} -type f -executable`; do
+  # Return code from grep is 1 if not found, 0 if found
+  ldd $file 2>&1 | grep "${searchfor}"
+  if [[ $? -eq 0 ]]; then
+    echo "Found ${searchfor} linked in ${file}"
+    ierror=1
+  fi
+done
+
+exit ${ierror}