Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update script for building for software.eessi.io (version 2023.06) #383

Closed
wants to merge 9 commits into from
Closed

update script for building for software.eessi.io (version 2023.06) #383

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
047c366
update script for building for software.eessi.io (version 2023.06)
boegel Nov 15, 2023
a72b077
add initial easystack file for software.eessi.io + move 'eb --show-co…
boegel Nov 17, 2023
4f3691b
add CI workflow for software.eessi.io + use software.eessi.io in test…
boegel Nov 17, 2023
3a9f86d
use archdetect by default for software.eessi.io version 2023.06
boegel Nov 20, 2023
fe65d9e
add debugging to CI workflow to test scripts
boegel Nov 20, 2023
2e4a43b
always let eessi_container.sh mount cvmfs-config repo to get access t…
boegel Nov 20, 2023
82328a1
Merge branch '2023.06-software.eessi.io' into 2023.06-eessi.io
boegel Nov 25, 2023
2a58fdd
Merge branch '2023.06-software.eessi.io' into 2023.06-eessi.io
boegel Nov 25, 2023
faa2f03
Merge branch '2023.06-software.eessi.io' into 2023.06-eessi.io
boegel Nov 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion EESSI-pilot-install-software.sh → EESSI-install-software.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/bin/bash
#
# Script to install EESSI pilot software stack (version set through init/eessi_defaults)
# Script to install EESSI software stack (version set through init/eessi_defaults)

# see example parsing of command line arguments at
# https://wiki.bash-hackers.org/scripting/posparams#using_a_while_loop
Expand Down
16 changes: 8 additions & 8 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
# Software layer

The software layer of the EESSI project uses [EasyBuild](https://easybuild.readthedocs.io), [Lmod](https://lmod.readthedocs.io) and [archspec](https://archspec.readthedocs.io).
The software layer of the EESSI project uses [EasyBuild](https://docs.easybuild.io), [Lmod](https://lmod.readthedocs.io) and [archspec](https://archspec.readthedocs.io).

See also https://eessi.github.io/docs/software_layer.
See also https://www.eessi.io/docs/software_layer .

## Pilot software stack

You can set up your environment by sourcing the init script:

```
$ source /cvmfs/pilot.eessi-hpc.org/versions/2023.06/init/bash
Found EESSI pilot repo @ /cvmfs/pilot.eessi-hpc.org/versions/2023.06!
$ source /cvmfs/software.eessi.io/versions/2023.06/init/bash
Found EESSI repo @ /cvmfs/software.eessi.io/versions/2023.06!
Derived subdirectory for software layer: x86_64/intel/haswell
Using x86_64/intel/haswell subdirectory for software layer (HARDCODED)
Using x86_64/intel/haswell subdirectory for software layer
Initializing Lmod...
Prepending /cvmfs/pilot.eessi-hpc.org/versions/2023.06/software/x86_64/intel/haswell/modules/all to $MODULEPATH...
Environment set up to use EESSI pilot software stack, have fun!
[EESSI pilot 2023.06] $
Prepending /cvmfs/software.eessi.io/versions/2023.06/software/x86_64/intel/haswell/modules/all to $MODULEPATH...
Environment set up to use EESSI (2023.06), have fun!
[EESSI 2023.06] $
```

### Accessing EESSI via a container
Expand Down
16 changes: 8 additions & 8 deletions bot/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,12 +128,12 @@ EESSI_REPOS_CFG_DIR_OVERRIDE=$(cfg_get_value "repository" "repos_cfg_dir")
export EESSI_REPOS_CFG_DIR_OVERRIDE=${EESSI_REPOS_CFG_DIR_OVERRIDE:-${PWD}/cfg}
echo "bot/build.sh: EESSI_REPOS_CFG_DIR_OVERRIDE='${EESSI_REPOS_CFG_DIR_OVERRIDE}'"

# determine pilot version to be used from .repository.repo_version in ${JOB_CFG_FILE}
# here, just set & export EESSI_PILOT_VERSION_OVERRIDE
# determine EESSI version to be used from .repository.repo_version in ${JOB_CFG_FILE}
# here, just set & export EESSI_VERSION_OVERRIDE
# next script (eessi_container.sh) makes use of it via sourcing init scripts
# (e.g., init/eessi_defaults or init/minimal_eessi_env)
export EESSI_PILOT_VERSION_OVERRIDE=$(cfg_get_value "repository" "repo_version")
echo "bot/build.sh: EESSI_PILOT_VERSION_OVERRIDE='${EESSI_PILOT_VERSION_OVERRIDE}'"
export EESSI_VERSION_OVERRIDE=$(cfg_get_value "repository" "repo_version")
echo "bot/build.sh: EESSI_VERSION_OVERRIDE='${EESSI_VERSION_OVERRIDE}'"

# determine CVMFS repo to be used from .repository.repo_name in ${JOB_CFG_FILE}
# here, just set EESSI_CVMFS_REPO_OVERRIDE, a bit further down
Expand Down Expand Up @@ -210,9 +210,9 @@ BUILD_TMPDIR=$(grep ' as tmp directory ' ${build_outerr} | cut -d ' ' -f 2)
TARBALL_STEP_ARGS+=("--resume" "${BUILD_TMPDIR}")

timestamp=$(date +%s)
# to set EESSI_PILOT_VERSION we need to source init/eessi_defaults now
# to set EESSI_VERSION we need to source init/eessi_defaults now
source init/eessi_defaults
export TGZ=$(printf "eessi-%s-software-%s-%s-%d.tar.gz" ${EESSI_PILOT_VERSION} ${EESSI_OS_TYPE} ${EESSI_SOFTWARE_SUBDIR_OVERRIDE//\//-} ${timestamp})
export TGZ=$(printf "eessi-%s-software-%s-%s-%d.tar.gz" ${EESSI_VERSION} ${EESSI_OS_TYPE} ${EESSI_SOFTWARE_SUBDIR_OVERRIDE//\//-} ${timestamp})

# value of first parameter to create_tarball.sh - TMP_IN_CONTAINER - needs to be
# synchronised with setting of TMP_IN_CONTAINER in eessi_container.sh
Expand All @@ -221,8 +221,8 @@ export TGZ=$(printf "eessi-%s-software-%s-%s-%d.tar.gz" ${EESSI_PILOT_VERSION} $
TMP_IN_CONTAINER=/tmp
echo "Executing command to create tarball:"
echo "./eessi_container.sh ${COMMON_ARGS[@]} ${TARBALL_STEP_ARGS[@]}"
echo " -- ./create_tarball.sh ${TMP_IN_CONTAINER} ${EESSI_PILOT_VERSION} ${EESSI_SOFTWARE_SUBDIR_OVERRIDE} /eessi_bot_job/${TGZ} 2>&1 | tee -a ${tar_outerr}"
echo " -- ./create_tarball.sh ${TMP_IN_CONTAINER} ${EESSI_VERSION} ${EESSI_SOFTWARE_SUBDIR_OVERRIDE} /eessi_bot_job/${TGZ} 2>&1 | tee -a ${tar_outerr}"
./eessi_container.sh "${COMMON_ARGS[@]}" "${TARBALL_STEP_ARGS[@]}" \
-- ./create_tarball.sh ${TMP_IN_CONTAINER} ${EESSI_PILOT_VERSION} ${EESSI_SOFTWARE_SUBDIR_OVERRIDE} /eessi_bot_job/${TGZ} 2>&1 | tee -a ${tar_outerr}
-- ./create_tarball.sh ${TMP_IN_CONTAINER} ${EESSI_VERSION} ${EESSI_SOFTWARE_SUBDIR_OVERRIDE} /eessi_bot_job/${TGZ} 2>&1 | tee -a ${tar_outerr}

exit 0
20 changes: 10 additions & 10 deletions bot/inspect.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -195,12 +195,12 @@ EESSI_REPOS_CFG_DIR_OVERRIDE=$(cfg_get_value "repository" "repos_cfg_dir")
export EESSI_REPOS_CFG_DIR_OVERRIDE=${EESSI_REPOS_CFG_DIR_OVERRIDE:-${PWD}/cfg}
echo "bot/inspect.sh: EESSI_REPOS_CFG_DIR_OVERRIDE='${EESSI_REPOS_CFG_DIR_OVERRIDE}'"

# determine pilot version to be used from .repository.repo_version in ${JOB_CFG_FILE}
# here, just set & export EESSI_PILOT_VERSION_OVERRIDE
# determine EESSI version to be used from .repository.repo_version in ${JOB_CFG_FILE}
# here, just set & export EESSI_VERSION_OVERRIDE
# next script (eessi_container.sh) makes use of it via sourcing init scripts
# (e.g., init/eessi_defaults or init/minimal_eessi_env)
export EESSI_PILOT_VERSION_OVERRIDE=$(cfg_get_value "repository" "repo_version")
echo "bot/inspect.sh: EESSI_PILOT_VERSION_OVERRIDE='${EESSI_PILOT_VERSION_OVERRIDE}'"
export EESSI_VERSION_OVERRIDE=$(cfg_get_value "repository" "repo_version")
echo "bot/inspect.sh: EESSI_VERSION_OVERRIDE='${EESSI_VERSION_OVERRIDE}'"

# determine CVMFS repo to be used from .repository.repo_name in ${JOB_CFG_FILE}
# here, just set EESSI_CVMFS_REPO_OVERRIDE, a bit further down
Expand Down Expand Up @@ -260,11 +260,11 @@ base_dir=$(dirname $(realpath $0))
# TODO better use script from tarball???
source ${base_dir}/../init/eessi_defaults

if [ -z $EESSI_PILOT_VERSION ]; then
echo "ERROR: \$EESSI_PILOT_VERSION must be set!" >&2
if [ -z $EESSI_VERSION ]; then
echo "ERROR: \$EESSI_VERSION must be set!" >&2
exit 1
fi
EESSI_COMPAT_LAYER_DIR="${EESSI_CVMFS_REPO}/versions/${EESSI_PILOT_VERSION}/compat/linux/$(uname -m)"
EESSI_COMPAT_LAYER_DIR="${EESSI_CVMFS_REPO}/versions/${EESSI_VERSION}/compat/linux/$(uname -m)"

# NOTE The below requires access to the CVMFS repository. We could make a first
# test run with a container. For now we skip the test.
Expand Down Expand Up @@ -295,8 +295,8 @@ fi
if [ ! -z ${EESSI_CVMFS_REPO_OVERRIDE} ]; then
echo "export EESSI_CVMFS_REPO_OVERRIDE=${EESSI_CVMFS_REPO_OVERRIDE}" >> ${RESUME_SCRIPT}
fi
if [ ! -z ${EESSI_PILOT_VERSION_OVERRIDE} ]; then
echo "export EESSI_PILOT_VERSION_OVERRIDE=${EESSI_PILOT_VERSION_OVERRIDE}" >> ${RESUME_SCRIPT}
if [ ! -z ${EESSI_VERSION_OVERRIDE} ]; then
echo "export EESSI_VERSION_OVERRIDE=${EESSI_VERSION_OVERRIDE}" >> ${RESUME_SCRIPT}
fi
if [ ! -z ${http_proxy} ]; then
echo "export http_proxy=${http_proxy}" >> ${RESUME_SCRIPT}
Expand Down Expand Up @@ -428,7 +428,7 @@ echo "Executing command to start interactive session to inspect build job:"
# TODO possibly add information on how to init session after the prefix is
# entered, initialization consists of
# - environment variable settings (see 'run_in_compat_layer_env.sh')
# - setup steps run in 'EESSI-pilot-install-software.sh'
# - setup steps run in 'EESSI-install-software.sh'
# These initializations are combined into a single script that is executed when
# the shell in startprefix is started. We set the env variable BASH_ENV here.
if [[ -z ${run_in_prefix} ]]; then
Expand Down
2 changes: 1 addition & 1 deletion check_missing_installations.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/bin/bash
#
# Script to check for missing installations in EESSI pilot software stack (version 2023.06)
# Script to check for missing installations in EESSI software stack (version 2023.06)
#
# author: Kenneth Hoste (@boegel)
# author: Thomas Roeblitz (@trz42)
Expand Down
4 changes: 2 additions & 2 deletions create_directory_tarballs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,9 @@ mkdir "${tartmp}/${version}"
tarname="eessi-${version}-init-$(date +%s).tar.gz"
curl -Ls ${SOFTWARE_LAYER_TARBALL_URL} | tar xzf - -C "${tartmp}/${version}" --strip-components=1 --no-wildcards-match-slash --wildcards '*/init/'
source "${tartmp}/${version}/init/minimal_eessi_env"
if [ "${EESSI_PILOT_VERSION}" != "${version}" ]
if [ "${EESSI_VERSION}" != "${version}" ]
then
fatal_error "Specified version ${version} does not match version ${EESSI_PILOT_VERSION} in the init files!"
fatal_error "Specified version ${version} does not match version ${EESSI_VERSION} in the init files!"
fi
tar czf "${tarname}" -C "${tartmp}" "${version}"
rm -rf "${tartmp}"
Expand Down
24 changes: 12 additions & 12 deletions create_tarball.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@ set -e
base_dir=$(dirname $(realpath $0))

if [ $# -ne 4 ]; then
echo "ERROR: Usage: $0 <EESSI tmp dir (example: /tmp/$USER/EESSI)> <pilot version (example: 2021.03)> <CPU arch subdir (example: x86_64/amd/zen2)> <path to tarball>" >&2
echo "ERROR: Usage: $0 <EESSI tmp dir (example: /tmp/$USER/EESSI)> <version (example: 2023.06)> <CPU arch subdir (example: x86_64/amd/zen2)> <path to tarball>" >&2
exit 1
fi
eessi_tmpdir=$1
pilot_version=$2
eessi_version=$2
cpu_arch_subdir=$3
target_tgz=$4

Expand All @@ -20,15 +20,15 @@ os="linux"
source ${base_dir}/init/eessi_defaults
cvmfs_repo=${EESSI_CVMFS_REPO}

software_dir="${cvmfs_repo}/versions/${pilot_version}/software/${os}/${cpu_arch_subdir}"
software_dir="${cvmfs_repo}/versions/${eessi_version}/software/${os}/${cpu_arch_subdir}"
if [ ! -d ${software_dir} ]; then
echo "Software directory ${software_dir} does not exist?!" >&2
exit 2
fi

overlay_upper_dir="${eessi_tmpdir}/overlay-upper"

software_dir_overlay="${overlay_upper_dir}/versions/${pilot_version}/software/${os}/${cpu_arch_subdir}"
software_dir_overlay="${overlay_upper_dir}/versions/${eessi_version}/software/${os}/${cpu_arch_subdir}"
if [ ! -d ${software_dir_overlay} ]; then
echo "Software directory overlay ${software_dir_overlay} does not exist?!" >&2
exit 3
Expand All @@ -40,22 +40,22 @@ echo ">> Collecting list of files/directories to include in tarball via ${PWD}..
files_list=${tmpdir}/files.list.txt
module_files_list=${tmpdir}/module_files.list.txt

if [ -d ${pilot_version}/software/${os}/${cpu_arch_subdir}/.lmod ]; then
if [ -d ${eessi_version}/software/${os}/${cpu_arch_subdir}/.lmod ]; then
# include Lmod cache and configuration file (lmodrc.lua),
# skip whiteout files and backup copies of Lmod cache (spiderT.old.*)
find ${pilot_version}/software/${os}/${cpu_arch_subdir}/.lmod -type f | egrep -v '/\.wh\.|spiderT.old' > ${files_list}
find ${eessi_version}/software/${os}/${cpu_arch_subdir}/.lmod -type f | egrep -v '/\.wh\.|spiderT.old' > ${files_list}
fi
if [ -d ${pilot_version}/software/${os}/${cpu_arch_subdir}/modules ]; then
if [ -d ${eessi_version}/software/${os}/${cpu_arch_subdir}/modules ]; then
# module files
find ${pilot_version}/software/${os}/${cpu_arch_subdir}/modules -type f | grep -v '/\.wh\.' >> ${files_list}
find ${eessi_version}/software/${os}/${cpu_arch_subdir}/modules -type f | grep -v '/\.wh\.' >> ${files_list}
# module symlinks
find ${pilot_version}/software/${os}/${cpu_arch_subdir}/modules -type l | grep -v '/\.wh\.' >> ${files_list}
find ${eessi_version}/software/${os}/${cpu_arch_subdir}/modules -type l | grep -v '/\.wh\.' >> ${files_list}
# module files and symlinks
find ${pilot_version}/software/${os}/${cpu_arch_subdir}/modules/all -type f -o -type l \
find ${eessi_version}/software/${os}/${cpu_arch_subdir}/modules/all -type f -o -type l \
| grep -v '/\.wh\.' | grep -v '/\.modulerc\.lua' | sed -e 's/.lua$//' | sed -e 's@.*/modules/all/@@g' | sort -u \
>> ${module_files_list}
fi
if [ -d ${pilot_version}/software/${os}/${cpu_arch_subdir}/software -a -r ${module_files_list} ]; then
if [ -d ${eessi_version}/software/${os}/${cpu_arch_subdir}/software -a -r ${module_files_list} ]; then
# installation directories but only those for which module files were created
# Note, we assume that module names (as defined by 'PACKAGE_NAME/VERSION.lua'
# using EasyBuild's standard module naming scheme) match the name of the
Expand All @@ -64,7 +64,7 @@ if [ -d ${pilot_version}/software/${os}/${cpu_arch_subdir}/software -a -r ${modu
# installation directories), the procedure will likely not work.
for package_version in $(cat ${module_files_list}); do
echo "handling ${package_version}"
ls -d ${pilot_version}/software/${os}/${cpu_arch_subdir}/software/${package_version} \
ls -d ${eessi_version}/software/${os}/${cpu_arch_subdir}/software/${package_version} \
| grep -v '/\.wh\.' >> ${files_list}
done
fi
Expand Down
14 changes: 7 additions & 7 deletions eessi_container.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ display_help() {
echo " -m | --mode MODE - with MODE==shell (launch interactive shell) or"
echo " MODE==run (run a script or command) [default: shell]"
echo " -r | --repository CFG - configuration file or identifier defining the"
echo " repository to use [default: EESSI-pilot via"
echo " repository to use [default: EESSI via"
echo " default container, see --container]"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The container images we use should probably be updated with the new configuration too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, yes and no.

Once cvmfs-contrib/config-repo#206 is merged, software.eessi.io should become automatically available with the current build (and client) container, I think.

So I prefer to try and see if that's the case first before we update the build container (which may be wise anyway, to sync up with the latest CernVM-FS for example).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we want to make use of the configuration provided by cvmfs-config-default, we do have to make sure to also mount /cvmfs/cvmfs-config.cern.ch in the container, as that provides the configuration files. So, we need to add a --fusemount container:cvmfs2 cvmfs-config.cern.ch /cvmfs/cvmfs-config.cern.ch to the apptainer command.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When trying interactively, for some reason accessing /cvmfs/cvmfs-config.cern.ch just works even without the extra --fusemount option, and hence so does accessing /cvmfs/software.eessi.io:

Apptainer> find /etc/cvmfs | grep eessi
/etc/cvmfs/domain.d/eessi-hpc.org.conf
/etc/cvmfs/keys/eessi-hpc.org
/etc/cvmfs/keys/eessi-hpc.org/pilot.eessi-hpc.org.pub
/etc/cvmfs/keys/eessi-hpc.org/cvmfs-config.eessi-hpc.org.pub
/etc/cvmfs/keys/eessi-hpc.org/ci.eessi-hpc.org.pub

Apptainer> find /etc/cvmfs | grep eessi.io
Apptainer>

Apptainer> ls /cvmfs/cvmfs-config.cern.ch
etc


Apptainer> find /cvmfs/cvmfs-config.cern.ch -type f  grep eessi.io | xargs sha256sum
find: paths must precede expression: `grep'
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  -
Apptainer> find /cvmfs/cvmfs-config.cern.ch -type f | grep eessi.io | xargs sha256sum
7d484216c59c00b81c9eb89e5ac511930d0e601a36cc2bdb106f51d148b19f14  /cvmfs/cvmfs-config.cern.ch/etc/cvmfs/domain.d/eessi.io.conf
d86b2190859f5124e78e0ff711c5a042be10d569350130a0f10a537eaacbc86b  /cvmfs/cvmfs-config.cern.ch/etc/cvmfs/keys/eessi.io/eessi.io.pub

Apptainer>
Apptainer> ls /cvmfs/software.eessi.io
host_injections  versions

Apptainer> attr -g revision /cvmfs/software.eessi.io
Attribute "revision" had a 2 byte value for /cvmfs/software.eessi.io:
25

Maybe loading one CernVM-FS repo is enough to also trigger a mount of the cvmfs-config.cern.ch?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify: accessing /cvmfs/software.eessi.io was working for me because we have CernVM-FS installed system-wide as well, so I guess the mount picks up on /cvmfs/cvmfs-config.cern.ch from the host OS.

If CernVM-FS is not installed in the host OS, then it doesn't work, so we do indeed need to explicitly mount /cvmfs/cvmfs-config.cern.ch via --fusemount.

echo " -u | --resume DIR/TGZ - resume a previous run from a directory or tarball,"
echo " where DIR points to a previously used tmp directory"
Expand Down Expand Up @@ -111,7 +111,7 @@ VERBOSE=0
STORAGE=
LIST_REPOS=0
MODE="shell"
REPOSITORY="EESSI-pilot"
REPOSITORY="EESSI"
RESUME=
SAVE=
HTTP_PROXY=${http_proxy:-}
Expand Down Expand Up @@ -194,7 +194,7 @@ set -- "${POSITIONAL_ARGS[@]}"

if [[ ${LIST_REPOS} -eq 1 ]]; then
echo "Listing available repositories with format 'name [source]':"
echo " EESSI-pilot [default]"
echo " EESSI [default]"
if [[ -r ${EESSI_REPOS_CFG_FILE} ]]; then
cfg_load ${EESSI_REPOS_CFG_FILE}
sections=$(cfg_sections)
Expand Down Expand Up @@ -226,7 +226,7 @@ fi

# TODO (arg -r|--repository) check if repository is known
# REPOSITORY_ERROR_EXITCODE
if [[ ! -z "${REPOSITORY}" && "${REPOSITORY}" != "EESSI-pilot" && ! -r ${EESSI_REPOS_CFG_FILE} ]]; then
if [[ ! -z "${REPOSITORY}" && "${REPOSITORY}" != "EESSI" && ! -r ${EESSI_REPOS_CFG_FILE} ]]; then
fatal_error "arg '--repository ${REPOSITORY}' requires a cfg file at '${EESSI_REPOS_CFG_FILE}'" "${REPOSITORY_ERROR_EXITCODE}"
fi

Expand Down Expand Up @@ -403,7 +403,7 @@ BIND_PATHS="${BIND_PATHS},${EESSI_TMPDIR}:${TMP_IN_CONTAINER}"
# set up repository config (always create directory repos_cfg and populate it with info when
# arg -r|--repository is used)
mkdir -p ${EESSI_TMPDIR}/repos_cfg
if [[ "${REPOSITORY}" == "EESSI-pilot" ]]; then
if [[ "${REPOSITORY}" == "EESSI" ]]; then
# need to source defaults as late as possible (see other sourcing below)
source ${TOPDIR}/init/eessi_defaults

Expand All @@ -427,7 +427,7 @@ else
# map { local_filepath -> container_filepath }
#
# repo_name_domain is the domain part of the repo_name, e.g.,
# eessi-hpc.org for pilot.eessi-hpc.org
# eessi.io for software.eessi.io
#
# where config bundle includes the files (-> target location in container)
# - default.local -> /etc/cvmfs/default.local
Expand Down Expand Up @@ -479,7 +479,7 @@ else
target=${cfg_file_map[${src}]}
BIND_PATHS="${BIND_PATHS},${EESSI_TMPDIR}/repos_cfg/${src}:${target}"
done
export EESSI_PILOT_VERSION_OVERRIDE=${repo_version}
export EESSI_VERSION_OVERRIDE=${repo_version}
export EESSI_CVMFS_REPO_OVERRIDE="/cvmfs/${repo_name}"
# need to source defaults as late as possible (after *_OVERRIDEs)
source ${TOPDIR}/init/eessi_defaults
Expand Down
4 changes: 2 additions & 2 deletions init/Magic_Castle/bash
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ EESSI_SILENT=1
source $(dirname "$BASH_SOURCE")/../eessi_environment_variables

# Don't change the default prompt
# export PS1="[EESSI pilot $EESSI_PILOT_VERSION] $ "
# export PS1="[EESSI $EESSI_VERSION] $ "

# Provide a clean MODULEPATH
export MODULEPATH_ROOT=$EESSI_MODULEPATH
Expand Down Expand Up @@ -36,4 +36,4 @@ else
module reload
fi

echo "Environment set up to use EESSI pilot software stack (${EESSI_PILOT_VERSION}), have fun!"
echo "Environment set up to use EESSI (${EESSI_VERSION}), have fun!"
8 changes: 4 additions & 4 deletions init/Magic_Castle/eessi_pilot_python3 → init/Magic_Castle/eessi_python3
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@ export EESSI_SILENT=1
# for MacOS due to the use of `readlink`)
source $(dirname "$(readlink -f "$BASH_SOURCE")")/../eessi_environment_variables

eessi_pilot_python=$(ls ${EESSI_SOFTWARE_PATH}/software/Python/3*GCCcore*/bin/python | sed 1q)
if [ -f "$eessi_pilot_python" ]; then
$eessi_pilot_python "$@"
eessi_python=$(ls ${EESSI_SOFTWARE_PATH}/software/Python/3*GCCcore*/bin/python | sed 1q)
if [ -f "$eessi_python" ]; then
$eessi_python "$@"
else
echo "ERROR: No EESSI pilot python 3 available."
echo "ERROR: No EESSI Python 3 available."
false
fi
6 changes: 3 additions & 3 deletions init/bash
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ source $(dirname "$BASH_SOURCE")/eessi_environment_variables
# only continue if setting EESSI environment variables worked fine
if [ $? -eq 0 ]; then

export PS1="[EESSI pilot $EESSI_PILOT_VERSION] $ "
export PS1="[EESSI $EESSI_VERSION] $ "

# add location of commands provided by compat layer to $PATH;
# see https://github.com/EESSI/software-layer/issues/52
Expand All @@ -28,12 +28,12 @@ if [ $? -eq 0 ]; then
module use $EESSI_MODULEPATH

#echo >> $output
#echo "*** Known problems in the ${EESSI_PILOT_VERSION} pilot software stack ***" >> $output
#echo "*** Known problems in the ${EESSI_VERSION} software stack ***" >> $output
#echo >> $output
#echo "1) ..." >> $output
#echo >> $output
#echo >> $output

echo "Environment set up to use EESSI pilot software stack, have fun!" >> $output
echo "Environment set up to use EESSI (${EESSI_VERSION}), have fun!"

fi
4 changes: 2 additions & 2 deletions init/eessi_defaults
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,5 @@
# license: GPLv2
#

export EESSI_CVMFS_REPO="${EESSI_CVMFS_REPO_OVERRIDE:=/cvmfs/pilot.eessi-hpc.org}"
export EESSI_PILOT_VERSION="${EESSI_PILOT_VERSION_OVERRIDE:=2023.06}"
export EESSI_CVMFS_REPO="${EESSI_CVMFS_REPO_OVERRIDE:=/cvmfs/software.eessi.io}"
export EESSI_VERSION="${EESSI_VERSION_OVERRIDE:=2023.06}"
6 changes: 3 additions & 3 deletions init/eessi_environment_variables
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,11 @@ function error() {
false
}

# set up minimal environment: $EESSI_PREFIX, $EESSI_PILOT_VERSION, $EESSI_OS_TYPE, $EESSI_CPU_FAMILY, $EPREFIX
# set up minimal environment: $EESSI_PREFIX, $EESSI_VERSION, $EESSI_OS_TYPE, $EESSI_CPU_FAMILY, $EPREFIX
source $EESSI_INIT_DIR_PATH/minimal_eessi_env

if [ -d $EESSI_PREFIX ]; then
echo "Found EESSI pilot repo @ $EESSI_PREFIX!" >> $output
echo "Found EESSI repo @ $EESSI_PREFIX!" >> $output

export EESSI_EPREFIX=$EPREFIX
if [ -d $EESSI_EPREFIX ]; then
Expand Down Expand Up @@ -82,5 +82,5 @@ if [ -d $EESSI_PREFIX ]; then
error "Compatibility layer directory $EESSI_EPREFIX not found!"
fi
else
error "EESSI pilot repository at $EESSI_PREFIX not found!"
error "EESSI repository at $EESSI_PREFIX not found!"
fi
4 changes: 2 additions & 2 deletions init/minimal_eessi_env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@
# $BASH_SOURCE points to correct path, see also http://mywiki.wooledge.org/BashFAQ/028
EESSI_INIT_DIR_PATH=$(dirname $(realpath $BASH_SOURCE))

# set up defaults: EESSI_CVMFS_REPO, EESSI_PILOT_VERSION
# set up defaults: EESSI_CVMFS_REPO, EESSI_VERSION
# script takes *_OVERRIDEs into account
source ${EESSI_INIT_DIR_PATH}/eessi_defaults

export EESSI_PREFIX=$EESSI_CVMFS_REPO/versions/$EESSI_PILOT_VERSION
export EESSI_PREFIX=$EESSI_CVMFS_REPO/versions/$EESSI_VERSION

if [[ $(uname -s) == 'Linux' ]]; then
export EESSI_OS_TYPE='linux'
Expand Down
2 changes: 1 addition & 1 deletion install_software_layer.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/bin/bash
base_dir=$(dirname $(realpath $0))
source ${base_dir}/init/eessi_defaults
./run_in_compat_layer_env.sh ./EESSI-pilot-install-software.sh "$@"
./run_in_compat_layer_env.sh ./EESSI-install-software.sh "$@"
Loading