Hi, I am currently compiling the MPAS-JEDI bundle on Derecho and I just did many times in the past but now that I got my account and resources back after a little period of having my account deactivated I am finding a problem.
I just follow the procedure in Building and compiling JEDI — JEDI Documentation 8.0.0 documentation that has always worked.
Now I find this error, seems like the file /usr/lib64/librt.so needed in the oops compilation does not exist but only /usr/lib64/librt.a exist.
I tried also to compile oops alone an the same error shows up. I don’t know if this can be an error related to some permission to see some files (because i think I have a third part user) but I do not think this is the case.
Do somebody had a similar problem? Is there some flag to compile oops with the static librt library?
Thanks for the support
Hi @antoniosandroni,
With a recent OS update on Derecho, the existing spack-strack libraries are affected. There exists some workaround which allows us keep using the existing spack-stack. Also there exists an effort to re-install spack-stack on Derecho.
Which version of stack-stack are you using ? Are you building the release v3 or develop branch of mpas-bundle ?
BJ
derecho% cat /etc/os-release
NAME="SLES"
VERSION="15-SP6"
VERSION_ID="15.6"
PRETTY_NAME="SUSE Linux Enterprise Server 15 SP6"
For most SLES 15 Service Packs (prior to SP6), the functionalities of libpthread are part of the main C library, libc.so, which is provided by the glibc package. Dynamic linking still works as expected using the -lpthread flag during compilation.
Specifically:
The actual shared library file is typically libpthread.so.0, which is part of the glibc package.
The developer files (.so symlinks for linking purposes) were previously provided by the glibc-devel package.
Starting with SLES 15 SP6, the libpthread, libdl, libutil, and libanl libraries were merged into the main libc library. The .so symlinks (like /usr/lib64/libpthread.so) are no longer provided by glibc-devel in SP6 and later, though the static .a files are still available for compatibility.
and
After upgrading to SLES 15 SP6, some libraries appear to be missing
A rebuild of spack-stack@1.9.3 has been shown to solve this problem, however a rebuild negatively impacts the UFS WM.
Follow here: [INSTALL]: builds using spack-stack 1.8.0 and 1.9.3 on derecho.hpc.ucar.edu no longer work #1815
Hi BJ
I am compiling the develop branch and in my gnu-derecho.sh script seems like I am using spack-stack-1.6.0
If I compile the v3 I have the same env setup script and the error during compilation is the same.
In addition in this case I have an error also during the cmake part and is the following
that I fix with the flag
AS
Thanks for the additional information, @rickgrubin-noaa. Also, thanks for rebuilding the spack-stack@1.9.3! I believe that Jim Wittig (@jim-p-w) is working on more realistic testcase and evaluation with mpas-jedi.
… however a rebuild negatively impacts the
UFS WM
As a quick question, is this … in the aspect of runtime & memory ? or forecast performance (forecast verification scoes such as BIAS, RMSE) ?
@antoniosandroni,
You can find the discussion on this issue from the pointed spack-stack github issue, and you may try the rebuilt spack-stack v1.9.3 (see here).
Or you may try the workaround with existing spack-stack.
In your cmake command, would you try following workaround (adding additional flags) ?
-DCMAKE_LIBRARY_PATH=/lib64 -Deckit_TARGETS_FILE=/glade/u/home/jwittig/spack-stack/eckit-targets.cmake
This has worked for me when I build the code with existing spack-stack v1.8.0 on Derecho. But I have not tested with spack-stack v1.6.0 myself.
Thank you,
BJ
and you may try the rebuilt spack-stack v1.9.3 (see here).
Please feel free to try the stacks (oneAPI and gcc) labeled rebuild-… but do not rely on them being in place with any amount of certainty – they may go away at any point in time, without notice.
Please also note that spack-stack@1.6.0 is no longer supported, and support for spack-stack@1.8.0 will likely cease at some point in (earlier than later) 2026.
I managed to compiled the v3 version with the 19.3 rebuild gcc yesterday using this spack-stack directory(I had some warnings yes but at least went to 100%)
/glade/work/epicufsrt/contrib/spack-stack/derecho/spack-stack-1.9.3/envs/rebuild-ue-gcc-12.4.0
but I noticed that yesterday such directory has been modified and the rebuild has been placed under a new folder /rebuild/
/glade/work/epicufsrt/contrib/spack-stack/derecho/spack-stack-1.9.3/envs/rebuild/rebuild-ue-gcc-12.4.0
and trying to compile, after changing my env setup script, I do not have the commands cmake and ecbuild even if I upload them as modules.
Do you had any similar problem?
Trying to solve it exporting the path of cmake and ecbuil to PATH during the cmake part seems like cmake does not find some libraries like
Seems like the way I solved the absence of the cmake command is not suitable…
The following are the compilation fails of both v3 and develop bundles with the spack-stack versions 18.0(using the cmake-targets trick) and the other rebuild combinations execpt for v3+rebuild19.3gcc that worked(up to /rebuild/ folder creation)
develop bundle with rebuild spack stack 19.3 gcc
develop bundle with rebuild spack stack 19.3 oneapi during the compilation I just have various warnings but not explicit errors but still fails)
v3 bundle with rebuild spack stack 19.3 oneapi the same of the develop rebuild oneapi, just warnings and fails
For the compilation with spack-stack 18.0 and the cmake-targets trick I have:
v3 bundle spack stack 18.0
develop bundle spack stack 18.0
Thanks for the help
Regards
AS
@antoniosandroni please use these envs:
glade/work/epicufsrt/contrib/spack-stack/derecho/spack-stack-1.9.3/envs/ue-oneapi-2024.2.1/glade/work/epicufsrt/contrib/spack-stack/derecho/spack-stack-1.9.3/envs/ue-gcc-12.4.0
The rebuild- envs were moved / made not available (as noted earlier) – the path change renders them unusable.
The ifort compiler warnings are harmless and can safely be ignored. Please also note that starting with spack-stack@v2, ifort will be replaced by ifx.
Lastly, please do not use spack-stack@1.8.0 – move all work to build against spack-stack@1.9.3, and plan for the new Intel Fortran compiler with spack-stack@v2.
Thank you!
I compiled the v3 bundle with the 1.9.3 gcc-12.4.0 spack-stack version and for now this works for me.
I did it in the standard way with no cmake-targets workaround, is this the correct way?
AS
@antoniosandroni, Yes, I think that you don’t need the workaround with newly installed spack-stack 1.9.3 !
Thanks you for your instruction, @rickgrubin-noaa !