undefined symbol: H5FD_subfiling_init

[justbennet]

We are trying to track down an error that we get.

Julia version 1.10.5 at location /gpfs1/sw/rh9/pkgs/julia/1.10.5/bin will be used.
Loading setup script for JuliaCall...
Finish loading setup script for JuliaCall.
Attempt 1 to run jl.prep()...
jl.prep() failed with error: Error happens in Julia.
InitError: could not load library "/users/c/d/cdrasher/.julia/artifacts/2117b531439d782e1cb0ebffcc4dcf11d274cc39/lib/libhdf5_fortran.so"
/users/c/d/cdrasher/.julia/artifacts/2117b531439d782e1cb0ebffcc4dcf11d274cc39/lib/../lib/libhdf5_f90cstub.so.310: undefined symbol: H5FD_subfiling_init
Stacktrace:
  [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
    @ Base.Libc.Libdl ./libdl.jl:117
  [2] dlopen(s::String, flags::UInt32)
    @ Base.Libc.Libdl ./libdl.jl:116
  [3] macro expansion
    @ ~/.julia/packages/JLLWrappers/GfYNv/src/products/library_generators.jl:63 [inlined]
  [4] __init__()
    @ HDF5_jll ~/.julia/packages/HDF5_jll/NQMjC/src/wrappers/x86_64-linux-gnu-libgfortran5-cxx11-mpi+mpich.jl:48
  [5] run_module_init(mod::Module, i::Int64)
    @ Base ./loading.jl:1134
  [6] register_restored_modules(sv::Core.SimpleVector, pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:1122
  [7] _include_from_serialized(pkg::Base.PkgId, path::String, ocachepath::String, depmods::Vector{Any})
    @ Base ./loading.jl:1067
  [8] _tryrequire_from_serialized(modkey::Base.PkgId, path::String, ocachepath::String, sourcepath::String, depmods::Vector{Any})
    @ Base ./loading.jl:1481
  [9] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128)
    @ Base ./loading.jl:1574
 [10] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1938
 [11] __require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1812
 [12] #invoke_in_world#3
    @ ./essentials.jl:926 [inlined]
 [13] invoke_in_world
    @ ./essentials.jl:923 [inlined]
 [14] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1803
 [15] macro expansion
    @ ./loading.jl:1790 [inlined]
 [16] macro expansion
    @ ./lock.jl:267 [inlined]
 [17] __require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1753
 [18] #invoke_in_world#3
    @ ./essentials.jl:926 [inlined]
 [19] invoke_in_world
    @ ./essentials.jl:923 [inlined]
 [20] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1746
 [21] eval
    @ ./boot.jl:385 [inlined]
 [22] eval_string(x::String)
    @ Main.JuliaCall /gpfs1/home/c/d/cdrasher/R/x86_64-pc-linux-gnu-library/4.4/JuliaCall/julia/setup.jl:203
 [23] docall(call1::Ptr{Nothing})
    @ Main.JuliaCall /gpfs1/home/c/d/cdrasher/R/x86_64-pc-linux-gnu-library/4.4/JuliaCall/julia/setup.jl:176
during initialization of module HDF5_jll

Looking at the libraries for the installed HDF5.jl, I find

$ pwd
/users/c/d/cdrasher/.julia/artifacts/2117b531439d782e1cb0ebffcc4dcf11d274cc39/lib
$ nm libhdf5_f90cstub.so.310.3.2 | grep H5FD_subfiling_init
libhdf5_f90cstub.so.310.3.2
                 U H5FD_subfiling_init
# nm libhdf5.so.310.5.1 | grep H5FD_subfiling_init
000000000017e000 T H5FD_subfiling_init

I use

# ldd libhdf5_f90cstub.so.310.3.2 | grep libhdf5.so
	libhdf5.so.310 => /gpfs1/home/c/d/cdrasher/.julia/artifacts/2117b531439d782e1cb0ebffcc4dcf11d274cc39/lib/./../lib/libhdf5.so.310 (0x00007f2f7c42c000)

followed by

$ nm libhdf5.so.310 | grep H5FD_subfiling_init
000000000017e000 T H5FD_subfiling_init

It seems that the symbol should be resolvable. Any thoughts on this?

Thanks, -- bennet

[mkitti]

There's some critical pieces of information here that I'm missing.

Where did /gpfs1/sw/rh9/pkgs/julia/1.10.5 come from? How did you install Julia?

It looks you like you are trying to run Julia from R on Linux.

Is it possible that there is another libhdf5 that is being loaded? Do you load any other packages in R before invoking JuliaCall? It is possible another libhdf5 is loaded which may not contain H5FD_subfiling_init.

I suggest setting the environment variable LD_DEBUG to either all or libs to view Linux loader debugging messages. See man 8 ld.so

Do you have the environment variable LD_LIBRARY_PATH or LD_PRELOAD set to something? This could preempt any kind of path.