Where to find current "specs" for building Spack-stack for skylab

JEDI
1

Hi,

I am looking at latest instructions for building a container for Spack Stack.

That’s all good except for the part that says ‘Edit config file and add the required specs in section “specs:”’

Previously, maybe a week or so ago, I found a page in the documentation for SpackStack or JEDI or somewhere else, that listed all of the “specs” to add. It listed the following:

  specs: [base-env@1.0.0, jedi-base-env@1.0.0, ewok-env@1.0.0, jedi-fv3-env@1.0.0,
    jedi-mpas-env@1.0.0, bacio@2.4.1, bison@3.8.2, bufr@12.0.0, ecbuild@3.7.2, eccodes@2.27.0, ecflow@5,
    eckit@1.24.4, ecmwf-atlas@0.35.0 +fckit +ectrans +tesselation +fftw, fiat@1.2.0, ectrans@1.2.0 +fftw,
    eigen@3.4.0, fckit@0.11.0, fms@release-jcsda, g2@3.4.5, g2tmpl@1.10.0, gftl-shared@1.6.1,
    gsibec@1.1.3, hdf@4.2.15, hdf5@1.14.0, ip@4.3.0, jasper@2.0.32, jedi-cmake@1.4.0,
    libpng@1.6.37, nccmp@1.9.0.1, netcdf-c@4.9.2, netcdf-cxx4@4.3.1,
    netcdf-fortran@4.6.0, nlohmann-json@3.10.5, nlohmann-json-schema-validator@2.1.0,
    parallelio@2.5.10, parallel-netcdf@1.12.2, py-eccodes@1.4.2, py-f90nml@1.4.3,
    py-gitpython@3.1.27, py-h5py@3.7.0, py-numpy@1.22.3,
    py-pandas@1.5.3, py-pip, py-pyyaml@5.4.1, py-scipy@1.9.3, py-shapely@1.8.0, py-xarray@2022.3.0,
    sp@2.3.3, udunits@2.2.28, w3nco@2.4.1, w3emc@2.10.0, nco@5.0.6, esmf@8.5.0, mapl@2.40.3,
    zlib@1.2.13, zstd@1.5.2, odc@1.4.6, shumlib@macos_clang_linux_intel_port,
    awscli@1.27.84, py-globus-cli@3.16.0]

That documentation, wherever it is, was critical to figuring out how to build the containers. But now, for reproducibility of my process, I need to find that documentation again. And I seem to have lost the browser tab and I can’t find it anywhere! I don’t remember where I found it. I’ve scraped documentation for JEDI and Spack Stack extensively in multiple versions.

Can someone please help point me to this? Also, a suggestion… If you’re going to have instructions that say to manually pasted in the specs, you probably should have a pointer to where to get that information.

2

Unbelievably, I just found it again. It’s not in the main documentation, it’s in a README in the spack stack repository: https://github.com/JCSDA/spack-stack/tree/release/1.5.1/configs/containers

All this time I’d been searching the docs, but I needed to look in the repo.

Maybe a note in the readthedocs documentation could have a little pointer to that README file in the configs/containers/README.md?

3

That’s outdated and replaced by an argument to the spack stack create ctr in the latest version of spack-stack develop: 7. Creating Containers — spack-stack 1.5.1 documentation

4

Dom,

I think there might be something wrong with the Documentation links. If I follow that link you posted, it says in the upper left that it is v1.5.1. But, at the bottom right in the black box with green text, it says “latest”. And when I click on the drop-down arrow in that box in the bottom right and select 1.5.1, the page changes to what I shared, and it still says version 1.5.1 in the upper left.

However, I can tell you that the instructions you pointed me to don’t work.

The link I shared shows the following:
spack stack create ctr docker-ubuntu-gcc-openmpi --template=empty

The link you shared shows this:
spack stack create ctr container=docker-ubuntu-gcc-openmpi --specs=jedi-ci

You’re right that the container is now just a plain argument, but the --specs argument always results in a container that has an un-concretizable spack spec. The only thing that works for me is to use the empty specs and manually add it in.

5

I’m going to try again from scratch, making sure to pay close attention to which version of spack-stack I’ve checked out (trying develop like you suggested) and make sure I’m looking at the docs version that exactly match the spack-stack version I’m using. I thought I’d done that before… but something seems off, so I’m going to try again.

6

So basically we merge the release branch back to develop but so far didn’t bother updating the “version” in doc/src/conf.py from the latest release tag to “develop”/“latest” and that’s where the 1.5.1 at the top left comes from). Worth giving this a try as it would help to avoid confusion (but note the section on pre-configured platforms will always have the information from the latest relaese once it’s merged back).

7

Ok. I was using version 1.5.1 before, which seems to want a command that looks like

spack stack create ctr docker-ubuntu-gcc-openmpi --template=empty

And then you manually fill in the specs using what’s in configs/containers/README.md .

That’s what I was doing before.

But, now I’ve gone back and had success with the new way using develop/latest instructions. And it does seem to work correctly.

So I think I’m good now. Thanks for the pointer, the new way seems easier.