patch reframe script to ensure Python packages are picked up from ReFrame installation directory in recent ReFrame easyconfigs #13844
Conversation
…rame installation directory in recent ReFrame easyconfigs
|
@boegelbot please test @ generoso |
|
@boegel: Request for testing this PR well received on generoso PR test command '
Test results coming soon (I hope)... - notification for comment with ID 907061074 processed Message to humans: this is just bookkeeping information for me, |
|
Test report by @boegel |
|
Test report by @boegelbot |
|
Test report by @branfosj |
| @@ -40,6 +40,9 @@ exts_list = [ | |||
| postinstallcmds = [ | |||
| "cp -r tutorials %(installdir)s", | |||
| "mkdir -p %(installdir)s/share && cp -r share/completions %(installdir)s/share/completions", | |||
| # patch reframe script to disable import of 'site' module to make sure that Python packages installed | |||
| # in ReFrame installation directory are used, rather than those in the Python installation | |||
| r"sed -i 's@/\(python[0-9.]*\)$@/\1 -S@g' %(installdir)s/bin/reframe", | |||
There was a problem hiding this comment.
I've checked and this change is causing the failure that I see. The Python being used is the one from my easybuild virtualenv.
diff on bin/reframe with and without this change.
< #!/rds/bear-apps/devel/eb-sjb-up/EL8/EL8-cas/venv-dev-develop-up/bin/python -S
---
> #!/rds/bear-apps/devel/eb-sjb-up/EL8/EL8-cas/venv-dev-develop-up/bin/python
There was a problem hiding this comment.
So adding the -S led to zipfile not being found anymore, because it's installed in your virtualenv?
There was a problem hiding this comment.
In the virtualenv
$ python -c "import zipfile; print(zipfile.__file__)"
/usr/lib64/python3.6/zipfile.py
There was a problem hiding this comment.
Would in this case help if you start python with -S, prepend external to the sys.path as is being done now and then call site.main()?
|
@boegel What about the 3.8.0 version? And is there a correct solution for this in the works? I need to get back to ReFrame stuff.... |
|
Test report by @akesandgren |
|
I get the same with the 3.8.0 version |
|
Adding setuptools to by build container I get this: missing stuff in the easyconfig? No, problem with the container having python3-gi installed due to a Recommends from another python package and python3-gi lacking a depends on pycairo with all that includes... |
|
@boegel You probably need this for 3.8.x too? |
|
I wouldn't do this for existing installations, as it might break reframe tests that depend on system installed packages. |
(created using
eb --new-pr)This fixes the problem I described in EESSI/software-layer#127.
For "normal" installations (not in an alternate sysroot like Gentoo Prefix) this fix is also relevant, to ensure that ReFrame picks up the Python packages as intended, rather than accidentally picking up stuff from the OS.
cc @vkarak, @victorusu, @teojgo