Update modules to spack-stack 1.8 and change pinned versions

[Dooruk]

Uses modules listed here:

https://github.com/JCSDA/spack-stack/wiki/Post%E2%80%90release-updates-for-spack%E2%80%90stack%E2%80%901.8.0

Which are located on Discover here:
/discover/nobackup/projects/gmao/advda/swell/jedi_modules

TODO:

• Compare results with Spack-stack 1.8 version against the Spack-stack 1.7 pinned version (November 19th)
  In other words:
  /discover/nobackup/projects/gmao/advda/swell/JediBundles/fv3_soca_SLES15_11192024_skylab8
  vs.
  /discover/nobackup/projects/gmao/advda/swell/JediBundles/fv3_soca_SLES15_11192024
  SOCA comparison provided identical results (same compilers) ✅

Skylab8 (JCSDA's workflow environment) corresponds to Spack-stack 1.8

FYI @rtodling @metdyn

[Dooruk]

@rtodling confirmed the results remained the same between the spack versions. @mranst we should start updating/installing software stacks (EVA, R2D2, pyinstaller) with the spack-stack 1.8 version. Did you mention today you installed swell with spack-stack 1.8 modules today?

[mranst]

@rtodling confirmed the results remained the same between the spack versions. @mranst we should start updating/installing software stacks (EVA, R2D2, pyinstaller) with the spack-stack 1.8 version. Did you mention today you installed swell with spack-stack 1.8 modules today?

One of the suites I tested worked without issue, but I'm having to fix some stuff with my python installation. Currently working on that

[Dooruk]

@rtodling confirmed the results remained the same between the spack versions. @mranst we should start updating/installing software stacks (EVA, R2D2, pyinstaller) with the spack-stack 1.8 version. Did you mention today you installed swell with spack-stack 1.8 modules today?

One of the suites I tested worked without issue, but I'm having to fix some stuff with my python installation. Currently working on that

Is it related to this by any chance?

GEOS-ESM/swell#406 (comment)

[Dooruk]

I installed solo and r2d2 in /discover/nobackup/projects/gmao/advda/JediOpt and py_lmod_installer in /discover/nobackup/projects/gmao/advda/swell/dev/src/py_lmod_installer.

We need to install EVA next. Are you able to install packages in /discover/nobackup/projects/gmao/advda/JediOpt/src/eva, what I mean is that do you have folder permissions @mranst ?

[mranst]

@rtodling confirmed the results remained the same between the spack versions. @mranst we should start updating/installing software stacks (EVA, R2D2, pyinstaller) with the spack-stack 1.8 version. Did you mention today you installed swell with spack-stack 1.8 modules today?

One of the suites I tested worked without issue, but I'm having to fix some stuff with my python installation. Currently working on that

Is it related to this by any chance?

GEOS-ESM/swell#406 (comment)

I had a different issue initially that I resolved, though it seems I am running up against this now. I'm a little confused on where this change needs to be made, is this a local installation issue or a thing in the shared modules?

[mranst]

I installed solo and r2d2 in /discover/nobackup/projects/gmao/advda/JediOpt and py_lmod_installer in /discover/nobackup/projects/gmao/advda/swell/dev/src/py_lmod_installer.

We need to install EVA next. Are you able to install packages in /discover/nobackup/projects/gmao/advda/JediOpt/src/eva, what I mean is that do you have folder permissions @mranst ?

I do not, do I need to be a member of advda?

[Dooruk]

I had a different issue initially that I resolved, though it seems I am running up against this now. I'm a little confused on where this change needs to be made, is this a local installation issue or a thing in the shared modules?

Local installation. In your installation folder there are /src, /core, and /modulefiles. In your modulefiles/core/*/ there are .lua files that defines folders and python versions for your LMOD modules. Python version is pythondir below, that used to be 3.10. Hope this helps

#sles15_skylab8.lua

help([[]])

local pkgName    = myModuleName()
local pkgVersion = myModuleVersion()
local pkgNameVer = myModuleFullName()

conflict(pkgName)

local opt = "/discover/nobackup/projects/gmao/advda/JediOpt/"
local pythondir = "python3.11"

local base = pathJoin(opt,"core",pkgNameVer)
local src = pathJoin(opt,"src",pkgNameVer)
prepend_path("PYTHONPATH", pathJoin(src,"src"))

prepend_path("PATH", pathJoin(base,"bin"))
prepend_path("PYTHONPATH", pathJoin(base,"lib",pythondir,"site-packages"))

whatis("Name: ".. pkgName)
whatis("Version: " .. pkgVersion)
whatis("Category: Software")
whatis("Description: JEDI BUNDLE")

I do not, do I need to be a member of advda?

Yeah, we should do that.

[mranst]

Is this the specific issue caused by that problem? When trying to run any t1test:

Lmod Warning:  Syntax error in file: /gpfsm/dswdev/jcsda/spack-stack/scu17/spack-stack-1.8.0/envs/ue-intel-2021.10.0/install/modulefiles/intel/2021.10.0/stack-intel-oneapi-mpi/2021.10.0.lua
 with command: setenv, one or more arguments are not strings.

While processing the following module(s):
    Module fullname                   Module Filename
    ---------------                   ---------------
    stack-intel-oneapi-mpi/2021.10.0  /gpfsm/dswdev/jcsda/spack-stack/scu17/spack-stack-1.8.0/envs/ue-intel-2021.10.0/install/modulefiles/intel/2021.10.0/stack-intel-oneapi-mpi/2021.10.0.lua

Typically I run swell using an interactive venv rather than as a module, so I'm not sure if that problem applies to me. I did try installing it as a module, ensuring that local python_dir = python3.11, and ended up getting the same message.

I'm assuming the issue is in one of these lines failing to os.getenv:

-- intel specific mpi wrapper environment variables
setenv("I_MPI_CC",  os.getenv("CC"))
setenv("I_MPI_CXX", os.getenv("CXX"))
setenv("I_MPI_F77", os.getenv("F77"))
setenv("I_MPI_F90", os.getenv("FC"))
setenv("I_MPI_FC",  os.getenv("FC"))

Not sure if it's related or an entirely different issue

[Dooruk]

Is this the specific issue caused by that problem? When trying to run any t1test:

Lmod Warning:  Syntax error in file: /gpfsm/dswdev/jcsda/spack-stack/scu17/spack-stack-1.8.0/envs/ue-intel-2021.10.0/install/modulefiles/intel/2021.10.0/stack-intel-oneapi-mpi/2021.10.0.lua
 with command: setenv, one or more arguments are not strings.

While processing the following module(s):
    Module fullname                   Module Filename
    ---------------                   ---------------
    stack-intel-oneapi-mpi/2021.10.0  /gpfsm/dswdev/jcsda/spack-stack/scu17/spack-stack-1.8.0/envs/ue-intel-2021.10.0/install/modulefiles/intel/2021.10.0/stack-intel-oneapi-mpi/2021.10.0.lua

Typically I run swell using an interactive venv rather than as a module, so I'm not sure if that problem applies to me. I did try installing it as a module, ensuring that local python_dir = python3.11, and ended up getting the same message.

I'm assuming the issue is in one of these lines failing to os.getenv:

-- intel specific mpi wrapper environment variables
setenv("I_MPI_CC",  os.getenv("CC"))
setenv("I_MPI_CXX", os.getenv("CXX"))
setenv("I_MPI_F77", os.getenv("F77"))
setenv("I_MPI_F90", os.getenv("FC"))
setenv("I_MPI_FC",  os.getenv("FC"))

Not sure if it's related or an entirely different issue

Hmm, I don't know if I will have a chance to look at this today. I can try installing swell on my end and see with spack 1.8.

Did you try with both installations? pylmod and uv? Also which module file are you using?

[mranst]

Hmm, I don't know if I will have a chance to look at this today. I can try installing swell on my end and see with spack 1.8.

Did you try with both installations? pylmod and uv? Also which module file are you using?

I tried both methods, though I typically invoke venv directly though python, rather than through uv, which hasn't caused me problems yet. I put in a draft PR to swell of what I'm doing, I also updated my mod_swell command to use the new spackstack version. I feel like I must be missing something wrong with my installation

[mranst]

I do not, do I need to be a member of advda?

Yeah, we should do that.

Can I just email support for that or does someone need to request it for me?