Failing submission to GPU partition in CI run

[laraPPr]

All rfm_job.sh fail to be submitted by reframe on the GPU partitions. with the following error.

sbatch: error: Invalid generic resource (gres) specification

However when I go to the Stage directory and run the command that failed the job gets submitted and runs. Their must be something that I'm missing that is set by ReFrame but not in the jobscript of launch command. That causes the job not to get submitted. I've already investigated it a little bit and I can't find what the difference might be.

ReFrame version: 4.7.3 & 4.7.2
Test-suite version: 0.5.1

[boegel]

@laraPPr Can you easily obtain the contents of the job script, and the exact sbatch command that is being used?

[laraPPr]

I did some further digging and the problem can't be with the job script itself, the test-suite or reframe

if reframe outside of the CI the error does not occur. And the test-suite runs fine.

So it is gonna be something that is picked up in the CI environment that causes the error. but I haven't tracked it down yet.

[laraPPr]

@laraPPr Can you easily obtain the contents of the job script, and the exact sbatch command that is being used?

Yes you can find the job script in the stage directory and the run command you can find in the logs but the problem is not with either of those.