Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial Commit Of New Installer Automated Build Process. #1086

Open
wants to merge 22 commits into
base: master
Choose a base branch
from

Conversation

steelhead31
Copy link
Contributor

@steelhead31 steelhead31 commented Dec 20, 2024

This PR includes the code needed to automate the production of the linux installer packages in jenkins:

The production version of this job is here : https://ci.adoptium.net/job/build-scripts/job/release/job/create_installer_linux/

It is triggered as a post build action by the release job, that publishes the release tarballs to the GitHub binary repos. ( https://ci.adoptium.net/job/build-scripts/job/release/job/refactor_openjdk_release_tool/ )

Process Overview:

The refactor release job above is triggered when the testing for a release is complete, and triggered from the release pipeline e.g ( https://ci.adoptium.net/job/build-scripts/job/release-openjdk17-pipeline/85/ )

The parameters utilised for this job from the publish links in the pipeline output, are then passed into the new create installer linux job

The above job will then create the relevant packages on a per version, architecture and platform ( linux or alpine linux ). 

Once the above process completes successfully, a downstream job will publish an archive containing the generate rpm specfiles, alpine apkbuild file or debian ( control, changelog and rules ) files which have been dynamically generated as part of the package build. The downstream job triggered is referenced in the pipeline for the create installer linux job, and is called publish_linux_pkg_src.   The downstream job published the pkg source files to version numbered releases in the following binary repo. https://github.com/adoptium/temurin-linux-pkg-sources

Development process in jenkins here : https://ci.adoptium.net/job/sfr-build-linux-package-modular/

Epic Link : #1005

See Also:

#1019
#1037
#1049
#1069
#1045
#1072
#1080
#1094

@sxa sxa self-requested a review December 20, 2024 09:54
@steelhead31 steelhead31 force-pushed the sfr_installer_automation branch 6 times, most recently from fd7ca81 to b95f731 Compare December 24, 2024 10:51
@steelhead31 steelhead31 force-pushed the sfr_installer_automation branch 3 times, most recently from 65fe376 to 5a8a40e Compare January 3, 2025 12:19
@jiekang
Copy link
Contributor

jiekang commented Jan 6, 2025

Can all the relevant GH issues be referenced here? It doesn't have to be the Resolves/Closes text.

@steelhead31 steelhead31 force-pushed the sfr_installer_automation branch from 09ca3cb to bed2bd0 Compare January 7, 2025 13:19
@steelhead31 steelhead31 force-pushed the sfr_installer_automation branch from fc0f0ee to cdd4718 Compare January 8, 2025 11:39
@steelhead31 steelhead31 force-pushed the sfr_installer_automation branch from 555b521 to 2c803a2 Compare January 9, 2025 18:59
@steelhead31 steelhead31 force-pushed the sfr_installer_automation branch from 4331f7e to 01cba03 Compare January 10, 2025 14:35
@steelhead31 steelhead31 force-pushed the sfr_installer_automation branch from 0dd9f36 to 10741a2 Compare January 13, 2025 08:48
@steelhead31 steelhead31 force-pushed the sfr_installer_automation branch from 59fc545 to a590f35 Compare January 13, 2025 11:34
Copy link
Member

@sxa sxa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added a few comments within the code - some of them are just suggestions, others stuff for the future. I haven't verified the technicaly functionality/command lines etc. or the template files at this time since I would expect any issues with those to have come out during your testing, but I may look at that later :-) A few overall comments:

  • This file is rather big!
  • You've got echo lines at the start of each phase. It would be good to have an "eyecatcher" prefix on each of those e.g. installer_stage: blah to be able to find the parts of the log, and potentially use it to extract timings. This is similar to build.sh : and openjdk_build_pipeline: (I think that's the right one!) that's in the build logs
  • I've made a couple of comments about indents in here when they jumped out at me, but the whole file probably just needs to have that checked before we commit it.
  • It may not make sense to try and change this, but for anyone else reviewing this we have quite a few different names of some of the variables e.g. arch vs PARCH vs packagearch vs packagearch{Deb,Rhel} vs {RHEL,SUSE}architecture and I wonder if we could have more uniformity in the naming of those.
    As I mentioned before the comments on the doc were made before I realised that was unchanged from the original, so feel free to disregard them for now (unless you're using those as the basis for new doc!)

linux_new/NOTES.txt Show resolved Hide resolved
linux_new/NOTES.txt Show resolved Hide resolved
linux_new/README.md Show resolved Hide resolved
linux_new/README.md Show resolved Hide resolved
linux_new/README.md Show resolved Hide resolved
linux_new/Jenkinsfile Outdated Show resolved Hide resolved
linux_new/Jenkinsfile Outdated Show resolved Hide resolved
linux_new/Jenkinsfile Outdated Show resolved Hide resolved
linux_new/Jenkinsfile Outdated Show resolved Hide resolved
linux_new/Jenkinsfile Outdated Show resolved Hide resolved
@steelhead31 steelhead31 force-pushed the sfr_installer_automation branch from 178f424 to e091e6e Compare January 15, 2025 18:55
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A block has been put on this Pull Request as this repository is temporarily under a code freeze due to an ongoing release cycle.

If this pull request needs to be merged during the release cycle then please comment /merge and a PMC member will be able to remove the block.

If the code freeze is over you can remove this block by commenting /thaw.

@steelhead31 steelhead31 force-pushed the sfr_installer_automation branch from 34fdfdd to 2a7bb83 Compare January 17, 2025 11:03
@steelhead31 steelhead31 force-pushed the sfr_installer_automation branch from ffe1e55 to 031c22b Compare January 17, 2025 15:24
@steelhead31 steelhead31 marked this pull request as ready for review January 20, 2025 09:11
@steelhead31 steelhead31 requested a review from sxa January 20, 2025 09:11
@gdams
Copy link
Member

gdams commented Jan 20, 2025

I'm not going to block this PR as it's clear that you've put a significant amount of work into this. My only concern is that I see a lot of J2 template files which feel duplicated in places and I wonder if we should centralise this. That said, this doesn't have to be done as part of this PR but I'd like us to keep this as an end goal if possible

@steelhead31
Copy link
Contributor Author

I'm not going to block this PR as it's clear that you've put a significant amount of work into this. My only concern is that I see a lot of J2 template files which feel duplicated in places and I wonder if we should centralise this. That said, this doesn't have to be done as part of this PR but I'd like us to keep this as an end goal if possible

Thanks @gdams I'll add a note to the EPIC to improve this element. The EPIC is tracking the future plans for improving this process beyond the initial automation attempt.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants