doc: make top level readme more consumable for first-time users

Signed-off-by: Stewart X Addison <[email protected]>

README.md

@@ -6,14 +6,168 @@ Eclipse Adoptium makes use of these scripts to build binaries on the build farm
 ## Repository contents
 
 This repository contains several useful scripts in order to build OpenJDK
-personally or at build farm scale.
+personally or at build farm scale via jenkins. For the Temurin project at
+Adoptium, this is done with the jenkins instance at https://ci.adotpium.net
 
 1. The `docs` folder contains images and utility scripts to produce up to date
 documentation.
 2. The `pipelines` folder contains the Groovy pipeline scripts for Jenkins
 (e.g. build | test | checksum | release).
 3. The `tools` folder contains `pipelines/` analysis scripts that deliever success/failure trends and build scripts for code-tool dependancies for the build and test process (e.g. asmtools | jcov | jtharness | jtreg | sigtest).
 
+## Overview of pipeline types
+
+The starting point on the jenkins instance from the perspective of the
+overall build pipelines is the
+[build-scripts folder](https://ci.adoptium.net/job/build-scripts/). This
+contains the top level pipelines which are used to run the different types
+of build. In the names in this document `XX` is the JDK version number e.g.
+8, 17, 21 and so on. There is one of these for each JDK version which we
+support.
+
+### openjdkXX-pipeline
+
+These were historically used for regular builds of each of our release
+platforms using the current state of the master branch of the codebase -
+which is it's default behaviour - but is now run each time there is a new
+tag in the upstream openjdk codebase.  These are triggered by the
+`betaTrigger_XXea` jobs in the
+[build-scripts/utils](https://ci.adoptium.net/job/build-scripts/job/utils/)
+folder.
+
+The betaTrigger_XXea jobs use
+[trigger_beta_build.groovy](https://github.com/adoptium/ci-jenkins-pipelines/blob/master/pipelines/build/common/trigger_beta_build.groovy)
+to determine when to run a build. This contains a trap for the expected GA
+release times to prevent triggering so that machine time is not used up
+while we are performing release build and test cycles. 
+
+Once complete, the openjdkXX-pipelines which will, by default, invoke the
+separate
+([refactor_openjdk_release_tool](https://ci.adoptium.net/job/build-scripts/job/release/job/refactor_openjdk_release_tool/))
+job which will publish them as an `ea-beta`-suffixed release in github  under e.g. 
+[temurin-21-binaries](https://github.com/adoptium/temurin21-binaries/releases?q=ea-beta&expanded=true}). 
+
+### release-openjdkXX-pipeline
+
+These are not publicly visible but are used to build the fully tested
+production binaries on a quarterly basis.  Similar to the openjdkXX-pipeline
+jobs these are automatically triggered by the releaseTrigger_jdkXX jobs in
+[build-scripts/utils](https://ci.adoptium.net/job/build-scripts/job/utils/)
+every time a new `-ga` suffixed tag is detected.  (TODO: Note on JDK8/arm32
+and potentially Win32)
+
+releaseTrigger_jdkXX runs once an hour between the 10th and 25th of release
+months (Jan, Mar, Apr, Jul, Sep, Oct) and check for a new `-ga` tag.  It runs
+[triggerReleasePipeline.sh](https://github.com/adoptium/mirror-scripts/blob/master/triggerReleasePipeline.sh)
+(from the [mirror-scripts](https://github.com/adoptium/mirror-scripts/)
+repository).  That script has a loop that checks 5 times with a ten minute
+gap between them so that the overall trigger is checked every ten minutes
+during the days when it is active based on checking for the "expected" tag
+from [releasePlan.cfg](https://github.com/adoptium/mirror-scripts/blob/master/releasePlan.cfg)
+using the readExpectedGATag function in
+[common.sh](https://github.com/adoptium/mirror-scripts/blob/master/common.sh)
+
+### evaluation-openjdkXX-pipeline
+
+These are similar to the openjdkXX-pipeline jobs, and are triggered from the
+same betaTrigger_XXea jobs.  The evaluation pipelines are for platforms
+which the Adoptium team are looking to potentially release at some point,
+but they are not yet reliable or sufficiently tested.
+which are not in the release.
+
+### weekly-openjdkXX-pipeline / weekly-evaluation-openjdkXX-pipeline
+
+These are no longer used. These were triggered over the weekend with an extended set
+of tests, but since the regular openjdkXX-pipeline jobs are now running
+approximately once a week (the usual cadence of new tags appearing in the
+upstream codebases) we are running the full AQA suite in those pipelines.
+These were triggered by timer and then invoked the openjdkXX-pipeline jobs
+with the appropriate parameters.
+
+### trestle-openjdkXX-pipeline
+
+Trestle is the name of the experimental project to allow upstream openjdk
+committers to run pipelines on our infrastructure in order to test code
+changes in openjdk on the full set of platforms which Temurin supports. They
+are triggered on demand from a subset of authorized users.
+
+### PR tester
+
+In addition to the main pipelines we have "PR tester" jobs that are run on
+PRs to the pipelines repository in order to ensure they do not have any
+unintended side effects before they are merged.  These are triggered when
+comments from authorized users are added into the PR comments. In that
+folder in jenkins there are separate versions of all of the
+openjdkXX-pipelines that can be used to run against PRs and will not
+"pollute" the history of the main pipelines.
+
+## Subjobs of the top level pipelines
+
+Each of the top level pipelines described above invoke lower level jobs to
+run the platform-specific builds.  The jenkins folders containing these
+scripts for each of the above top level pipelines are as follows:
+
+Top level pipeline | Platform-specific pipeline folder (TODO: Name these!)
+---|---
+openjdkXX-pipeline | [jobs/jdkXX](https://ci.adoptium.net/job/build-scripts/job/jobs/)
+evaluation-openjdkXX-pipeline | [jobs/evaluation/jdkXX](https://ci.adoptium.net/job/build-scripts/job/jobs/job/evaluation/) [†]
+weekly-openjdkXX-pipeline | jobs/jdkXX (Shared with openjdkXX-pipeline)
+release-openjdkXX-pipeline | [jobs/release/jobs/jdkXX](https://ci.adoptium.net/job/build-scripts/job/jobs/job/release/job/jobs/) (Restricted access)
+PR testers | build-test/jobs/jdkXX
+
+[†] - The release jobs here are restricted access.  The release folder here
+should also not be confused with the build-scripts/release folder which
+contains jobs used for the final publishing of the builds (early access or
+GA) to github
+
+*Note: jdkXX is generally related to the name of the upstream codebase, which
+will often have a `u` suffix.  At the moment we have a separate set of jobs
+for non-u and u versions when the upstream codebase changes.  TODO: Add note
+about the new branch process for jdk23+*
+
+Inside the jdkXX folders there are pipelines which perform a build of one
+variant (e.g.  Temurin) for on JDK version on one platform, for example
+[jdk21u-linux-aarch64-temurin](https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk21u/job/jdk21u-linux-aarch64-temurin/)
+which are reponsible for running the build and initiating the tests and
+other jobs against the completed build if successful.  A "Smoke Test" job
+such as
+[jdk21u-linux-aarch64-temurin-SmokeTests](https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk21u/job/jdk21u-linux-aarch64-temurin_SmokeTests/)
+(TODO: Links to info about what that contains) which is initiated after the
+build perfoms some basic tests against the build artefacts and acts as a
+gate to kicking off the subsequent steps.  Once complete, the
+openjdkXX-pipelines which run the early access builds will generally invoke
+the jobs to publish them as a release in github (e.g. 
+[temurin-21-binaries](https://github.com/adoptium/temurin21-binaries/releases?q=ea-beta&expanded=true}).
+
+## Job generation
+
+As you can see from the above sections, there are a lot of separate jobs in
+jenkins which are used during the build process.  These are not created
+manually, but are autogenerated using the generator pipelines.  TODO: More
+information on this is at ...
+
+The top level
+[build-pipeline-generator](https://ci.adoptium.net/job/build-scripts/job/utils/job/build-pipeline-generator/)
+job uses
+[build_pipeline_generator.groovy](https://github.com/adoptium/ci-jenkins-pipelines/blob/master/pipelines/build/regeneration/build_pipeline_generator.groovy)
+to generate the pipelines.  It will generate the top level
+openjdkXX-pipeline jobs.  Similarly there are pipeline_jobs_generator_jdkXX
+jobs which use
+[build_job_generator.groovy](pipelines/build/regeneration/build_job_generator.groovy)
+to generate the subjobs for each platform/variant combination.  Both of
+these pipelines are triggered on a change (PR merge) to the
+ci-jenkins-pipelines repository
+
+Similarly there is an evaluation-pipeline-generator and
+evaluation-pipeline_jobs_generator_jdkXX for generating the evaluation jobs,
+a trestle-pipeline-generator for those jobs, plus release-pipeline-generator
+andand release_pipeline_jobs_generator_jdkXX for release jobs (the release
+generators are not triggered automatically but are re-run manually at
+certain points during each release cycle
+
+For more details on the processm see the
+[regeneration documentation](https://github.com/adoptium/ci-jenkins-pipelines/blob/master/pipelines/build/regeneration/README.md)
+
 ## Configuration Files
 
 The [pipelines/jobs/configurations](pipelines/jobs/configurations) directory contains two categories of configuration files that our jenkins pipelines use (Nicknamed [#Build Configs](#build) and [#Nightly Configs](#nightly) for short).