Add reproducible comparison tests on windows

[sophia-guo]

Issues:

• Ansible request for jq for test machines infrastructure#3453
• Auto trigger by upstream( beta-trigger) , figure out the jdk and sbom file - adding reproducible.mk
• workdir doesn't work with jenkins user,
• install apt-cygwin - no permission @steelhead31 if apt-cygwin is must we will have to ask infra to install it on the jenkins agents?
• Invalid build.sh option: --with-version-opt
• configure: error: Your base path is too long. It is 170 characters long, but only 100 is supported.
• Error: Cannot locate spec.gmk, given by SPEC=win_x86-64. Add reproducible comparison tests on windows #3711 (comment)

[steelhead31]

@sophia-guo the requirement for apt-cygwin is no longer relevant, as the playbooks have been updated to install jq , so there is no need for this... I'll put a PR in for this update. PR here.. #3718

[sophia-guo]

Invalid build.sh option: --with-version-opt

14:11:15  Building JDK...
14:11:15  Starting ./makejdk-any-platform.sh to configure, build (Adoptium)OpenJDK binary
14:11:16  Parsing opt: --clean-git-repo
14:11:16  Possible opt arg: --jdk-boot-dir
14:11:16  Parsing opt: --jdk-boot-dir
14:11:16  Possible opt arg: /cygdrive/c/jenkins/workspace/Grinder/aqa-tests/TKG/output_17110446512142/Rebuild_Same_JDK_Reproducibility_Test_win_0/cmp20240321181053/jdk-20.0.2+9
14:11:16  Parsing opt: --with-version-opt
14:11:16  Possible opt arg: ea--target-file-name
14:11:16  Invalid build.sh option: --with-version-opt
14:11:16  cp: cannot stat '/cygdrive/c/jenkins/workspace/Grinder/aqa-tests/TKG/output_17110446512142/Rebuild_Same_JDK_Reproducibility_Test_win_0/cmp20240321181053/temurin-

https://ci.adoptium.net/view/Test_grinder/job/Grinder/9240/console

@steelhead31 idea?

[sophia-guo]

configure: error: Your base path is too long. It is 170 characters long, but only 100 is supported

Top dir

I'm at /cygdrive/c/jenkins/workspace/Grinder/aqa-tests/TKG/output_17113746753551/Rebuild_Same_JDK_Reproducibility_Test_win_0/cmp20240325135117/temurin-build/workspace/build/src

JDK build top dir:

16:06:01  Currently at '/cygdrive/c/workspace/openjdk-build/workspace/build/src'

This is specific to windows. In beta jdk build the TOPDIR is set as /cygdrive/c/workspace/openjdk-build, which is nothing related with the jenkins build job name. See the linux top dir is /home/jenkins/workspace/build-scripts/jobs/jdk21u/jdk21u-linux-x64-temurin/workspace/build/openjdkbuild. That is for windows due to the path lenght the topdir has been explicitly set to avoid this. In reproducible script it might also need to think about. @steelhead31 @andrew-m-leonard

[steelhead31]

configure: error: Your base path is too long. It is 170 characters long, but only 100 is supported

Top dir

I'm at /cygdrive/c/jenkins/workspace/Grinder/aqa-tests/TKG/output_17113746753551/Rebuild_Same_JDK_Reproducibility_Test_win_0/cmp20240325135117/temurin-build/workspace/build/src

JDK build top dir:

16:06:01  Currently at '/cygdrive/c/workspace/openjdk-build/workspace/build/src'

This is specific to windows. In beta jdk build the TOPDIR is set as /cygdrive/c/workspace/openjdk-build, which is nothing related with the jenkins build job name. See the linux top dir is /home/jenkins/workspace/build-scripts/jobs/jdk21u/jdk21u-linux-x64-temurin/workspace/build/openjdkbuild. That is for windows due to the path lenght the topdir has been explicitly set to avoid this. In reproducible script it might also need to think about. @steelhead31 @andrew-m-leonard

I think this is why I explicitly set a path from root originally , before I made the recent update to make it relative to the current working directory.

[sophia-guo]

def workspace = 'C:/workspace/openjdk-build/'

[steelhead31]

I could update my script to have the WORK_DIR passed as the 3rd parameter, would that help ?

[sophia-guo]

I think we might just set similar to def workspace = 'C:/workspace/openjdk-build/'. As long as it's not the root one which jenkins doesn't have the permission. And then the final reproduced zip file should be moved to WORK_DIR so aqa-tests can have access for the result files.

[steelhead31]

My script creates these 2 files..

Original JDK : "$WORK_DIR/src_jdk_dist.zip"
BuiltJDK : "$WORK_DIR/built_jdk.zip"

and the results of the comparison also finish up here..

"$WORK_DIR/compare/repro_diff.out"

Let me know if you'd like that changing, should be just some very minor amendments to my script

[sophia-guo]

@steelhead31 I think that should work

[andrew-m-leonard]

@sophia-guo yeah I am not surprised by Windows, there is fudge for ci-jenkins-pipelines to do the same:
https://github.com/adoptium/ci-jenkins-pipelines/blob/d966e612efa09a83175debe08fa187c2d07e9c2c/pipelines/build/common/openjdk_build_pipeline.groovy#L2042

[steelhead31]

@sophia-guo the PR for this script has now been merged :)

[sophia-guo]

@steelhead31 I didn't see the update of the WORKDIR ( working dir for build jdk) ? Build_JDK() . WORK_DIR="$CURR_DIR/cmp$(date +%Y%m%d%H%M%S)" will cause the configure: error: 'Your base path is too long ' in jenkins test environment.

[steelhead31]

@steelhead31 I didn't see the update of the WORKDIR ( working dir for build jdk) ? Build_JDK() . WORK_DIR="$CURR_DIR/cmp$(date +%Y%m%d%H%M%S)" will cause the configure: error: 'Your base path is too long ' in jenkins test environment.

I'll take a look...

[steelhead31]

@sophia-guo here is a PR, #3734  not sure what happened to the first commit... Now uses "C:/workspace/comp-jdk-build/" as a workdir

[sophia-guo]

Error: Cannot locate spec.gmk, given by SPEC=win_x86-64.
14:29:08 /cygdrive/c/comp-jdk-build/temurin-build/workspace/build/src/make/Init.gmk:124: *** Cannot continue. Stop.
https://ci.adoptium.net/view/Test_grinder/job/Grinder/9299/console

14:29:08  make[5]: Entering directory '/cygdrive/c/comp-jdk-build/temurin-build/workspace/build/src'
14:29:08  Error: Cannot locate spec.gmk, given by SPEC=win_x86-64.
14:29:08  /cygdrive/c/comp-jdk-build/temurin-build/workspace/build/src/make/Init.gmk:124: *** Cannot continue.  Stop.
14:29:08  make[5]: Leaving directory '/cygdrive/c/comp-jdk-build/temurin-build/workspace/build/src'
14:29:08  OpenJDK make failed, archiving make failed logs
14:29:08  Archiving and compressing with gzip
14:29:08  Your archive was created as /cygdrive/c/comp-jdk-build/temurin-build/workspace/build/src/build/windows-x86_64-server-release/OpenJDK.zip
14:29:08  Moving the artifact to location /cygdrive/c/comp-jdk-build/temurin-build/workspace/target//OpenJDK21U-makefailurelogs_x64_windows_hotspot_21.0.2_13.zip
14:29:08  archive done.
14:29:08  Failed to make the JDK, exiting
14:29:08  cp: cannot stat '/cygdrive/c/comp-jdk-build/temurin-build/workspace/target/OpenJDK21U-jdk_x64_windows_hotspot_21.0.2_13.zip': No such file or directory
1

[steelhead31]

@sophia-guo I think the issue is the path again... have you definitely picked up my changes to fix the windows_repro_build_compare.sh script working directory... ?
 

Line 43:
WORK_DIR="/cygdrive/c/comp-jdk-build"
 

14:56:42  configure: error: Your base path is too long. It is 170 characters long, but only 100 is supported 14:56:42  configure exiting with result code 1 14:56:42  make[5]: Entering directory '/cygdrive/c/jenkins/workspace/Grinder/aqa-tests/TKG/output_17113746753551/Rebuild_Same_JDK_Reproducibility_Test_win_0/cmp20240325135117/temurin-build/workspace/build/src' 14:56:42  Error: Cannot locate spec.gmk, given by SPEC=win_x86-64.

[sophia-guo]

@steelhead31 I believe i picked up the changes. The grinder was not there anymore. I will rebase and rerun.