Build fv3-jedi outside of jedi-bundle

JEDI
1

Building fv3-jedi outside of jedi-bundle

-- ---------------------------------------------------------

-- Build summary

-- ---------------------------------------------------------

-- system : [host] [Linux-5.15.0-1088-azure] [linux.64]

-- processor        : [x86_64]

-- endiness         : Little Endian -- IEEE []

-- build type       : [Release]

-- timestamp        : [20251203175550]

-- install prefix   : [/path/to/fv3-jedi-develop]

--   bin dir        : [/path/to/fv3-jedi-develop/bin]

--   lib dir        : [/path/to/fv3-jedi-develop/lib]

--   include dir    : [/path/to/fv3-jedi-develop/include]

--   data dir       : [/path/to/fv3-jedi-develop/share/fv3jedi]

--   cmake dir      : [/path/to/fv3-jedi-develop/lib/cmake/fv3jedi]

-- ---------------------------------------------------------

-- C -- IntelLLVM 2024.2.1

--     compiler   : /path/to/oneapi/icx

--     flags      :  -O3 -DNDEBUG  

--     link flags : 

-- CXX -- IntelLLVM 2024.2.1

--     compiler   : /path/to/oneapi/icpx

--     flags      :  -O3 -DNDEBUG  

--     link flags : 

-- Fortran -- Intel 2021.1.0.20240703

--     compiler   : /path/to/oneapi/ifx

--     flags      :  -O3 -DNDEBUG -unroll -inline -heap-arrays 32  

--     link flags : 

-- linker : /path/to/ld

-- ar     : /usr/bin/ar

-- ranlib : /usr/bin/ranlib

-- link flags

--     executable [    -Wl,--disable-new-dtags ]

--     shared lib [ -Wl,--disable-new-dtags ]

--     static lib [ -Wl,--disable-new-dtags ]

-- install rpath  : /path/to/fv3-jedi-develop/lib /path/to/fv3-jedi-develop/lib64 $ORIGIN/../lib

-- common definitions: ENABLE_GSIBEC=1

-- ---------------------------------------------------------

-- Feature summary

-- ---------------------------------------------------------

-- The following features have been enabled:

 * TESTS, fv3jedi: Enable the unit tests

-- The following OPTIONAL packages have been found:


 * Git

 * jedicmake

 * MKL

 * PkgConfig

 * Perl


-- The following REQUIRED packages have been found:


 * ecbuild (required version >= 3.6)

 * atlas (required version >= 0.35.0)

 * BLAS

 * boost_headers (required version == 1.84.0)

 * oops (required version >= 1.10.0)

 * saber (required version >= 1.10.0)

 * crtm (required version >= 2.2.3)

 * gsibec (required version >= 1.2.1)

 * sp

 * Threads

 * ufo (required version >= 1.10.0)

 * vader (required version >= 1.7.0)

 * OpenMP

 * MPI

 * NetCDF

 * FMS (required version >= 2023.04)

 * fv3jedilm (required version >= 1.5.0)


-- The following OPTIONAL packages have not been found:


 * ropp-ufo


-- Configuring done (46.0s)

CMake Error at /path/to/ioda-develop/lib/cmake/ioda/ioda-targets.cmake:105 (set_target_properties):

  The link interface of target "ioda" contains:

    NetCDF::NetCDF_CXX

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.

    * A find_package call is missing for an IMPORTED target.

    * An ALIAS target is missing.

ioda is built against netcdf-cxx (which is distinct from netcdf-cxx4).

Unclear on how to proceed given the above error.

2

Hi @rickgrubin-noaa, we recently repaired a similar looking issue with a Met Office build process. The fix was to make sure that the FindNetCDF from jedi-cmake was being used. jedi-cmake uses the NOAA CMakeModules repo which supplies the FindNetCDF script:

You’ve already got jedicmake in the list you posted, and the fix we made was to move the find_package command for netCDF from before the find_package for jedicmake to after. This is what we have now:

find_package( jedicmake REQUIRED )
find_package( NetCDF REQUIRED COMPONENTS C Fortran CXX ) # Prefer find NetCDF by jedicmake

If you are already doing this, then could you post any more (if any) details/messages about this CMake error.

Thanks!

3

Hi @stephenh

If’ I’m not mistaken, what you’ve shown with respect to find_package() is from ioda@develop / CMakeLists.txt – true? If so, that’s what I see when building ioda, and that build succeeds.

For fv3-jedi:

fv3-jedi@develop / CMakeLists.txt

################################################################################
# Dependencies
################################################################################

# jedi-cmake
find_package( jedicmake QUIET )

# NetCDF
find_package( NetCDF REQUIRED COMPONENTS C Fortran )

and as shown in the original post, jedicmake is optional here (and is found), an NetCDF is required – though just the C and Fortran components, NOT the C++ component – and, further the requests to find said packages are in the preferential order you mention.

The error messages posted are all that appear during the build process.

The question is why, after fv3-jedi successfully configures, is a cmake file from ioda that is looking for NetCDF_CXX playing a part in the build for fv3-jedi?

And ancillary to that: why does ioda require both netcdf-cxx (as evidenced by the aforementioned cmake file) to build? Is it somehow using both netcdf-cxx AND netcdf-cxx4?

4

@rickgrubin-noaa thanks for the info. I think there is a fault in the ioda CMake configuration where the NetCDF components are being set as PUBLIC when they should be PRIVATE. I’m pretty sure (need to verify) IODA does not export any of the netcdf APIs in its libraries. I think this is causing the confusion and issues with the find_package commands.

IODA should be fine with just the netcdf-cxx4 library. We have only the netcdf-cxx4 library installed and that works. Note that the CXX component of the NetCDF package should be able to use the netcdf_cxx4 library installation. The key is to have it properly find the ncxx4-config script (which is in the bin directory underneath the directory netcdf_cxx4 is installed in) and use that script to set up paths to the proper include and library files. I think that is one of the reasons we need the CMakeModule FindNetCDF script.

You are building ioda and installing it somewhere. Then in a subsequent build you do a find_package on ioda, while building fv3-jedi. Is this correct?

If I have the picture right, I think for now you will need to do a find_package( NetCDF REQUIRED COMPONENTS C Fortran CXX ) in the fv3-jedi build to get things working. Once we get the ioda fault fixed, then you shouldn’t need the CXX component in the fv3-jedi find_package.

Hope this helps.

5

I’ve created an issue in IODA to fix the netcdf dependencies (ie, make them private): https://github.com/JCSDA-internal/ioda/issues/1585. I’m hoping to get to this next week.

6

@stephenh thanks for the discussion.

A different conversation you had with Mark M. about ioda helped make clear that netcdf-cxx4 is a requirement to build, given that bin/nccx4-config must be found in order to properly configure. The CMakeModule/Modules/FindNetCDF.cmake file could live in a few places; it’s fine that jedi-cmake uses EMC’s repo as a submodule to reference it.

Regarding ioda and its configuration setting required components as PUBLIC vs. PRIVATE, we’ll look forward to a future fix.

You are building ioda and installing it somewhere. Then in a subsequent build you do a find_package on ioda, while building fv3-jedi. Is this correct?

Not quite; fv3-jedi does not look for ioda, either via CMake find_package() or (for example) spack depends_on(). I think that references to ioda are looked for via dependencies on other JEDI packages (at the moment, I’d bet on oops).

If I have the picture right, I think for now you will need to do a find_package( NetCDF REQUIRED COMPONENTS C Fortran CXX ) in the fv3-jedi build to get things working. Once we get the ioda fault fixed, then you shouldn’t need the CXX component in the fv3-jedi find_package.

This suggestion does work, thanks.