Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: rename bootstrap to merge_repository #8832

Merged
merged 8 commits into from
Dec 1, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Bootstrap
name: Merge Repository
on:
schedule:
- cron: '0 1 * * *' # nightly at 1 am UTC
Expand All @@ -19,19 +19,19 @@ jobs:
steps:
- uses: actions/checkout@v3
- name: Generate
run: ./generation/bootstrap.sh
run: ./generation/merge_repository.sh
env:
USERNAME: ${{ github.actor }}
- name: Compile
working-directory: generation/monorepo/google-cloud-java
working-directory: generation/monorepo/google-cloud-java-merged
run: mvn compile -T C1 -B
- name: Unit Tests
working-directory: generation/monorepo/google-cloud-java
working-directory: generation/monorepo/google-cloud-java-merged
run: mvn test -T C1 -B
- name: Push monorepo to branch
if: ${{ github.event_name != 'pull_request' }}
run: |
cd generation/monorepo/google-cloud-java
cd generation/monorepo/google-cloud-java-merged
git remote add monorepo https://${{ github.actor }}:${{ github.token }}@github.com/${{ github.repository }}.git
git push -f monorepo main:bootstrap_output
- name: Push diff to branch
Expand Down
9 changes: 5 additions & 4 deletions generation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,14 @@
## Using scripts independently

Most of the scripts in this directory can be used independently to do file updates across all modules.
They are also used and tested by `bootstrap.sh`.
They are also used and tested by `merge_repository.sh`.

## Generating the monorepo
## Merge repository into the monorepo

Built by running [bootstrap.sh](bootstrap.sh).
Built by running [merge_repository.sh](merge_repository.sh).

The script creates a new Git repository in `monorepo/google-cloud-java`.
The script creates a new Git repository in `monorepo/google-cloud-java` by merging
the repositories in the input.
After running the script locally, you can `cd` into the generated repository and build the project.

```shell
Expand Down
34 changes: 20 additions & 14 deletions generation/bootstrap.sh → generation/merge_repository.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,18 @@ cd "$(dirname "$0")"
rm -rf monorepo
mkdir monorepo

cp bootstrap.sh monorepo
cp merge_repository.sh monorepo
cp repos.txt monorepo

cd monorepo

git clone https://github.com/newren/git-filter-repo.git
export PATH=$PATH:`pwd`/git-filter-repo

mkdir google-cloud-java
merged_repository=google-cloud-java-merged
git clone https://github.com/googleapis/google-cloud-java "${merged_repository}"

cd google-cloud-java
git init -b main
cd "${merged_repository}"

cat ../repos.txt | while read service
do
Expand Down Expand Up @@ -51,7 +51,7 @@ do
git config --add secrets.allowed "dest.*src"
git commit -am "chore: setup owlbot configuration"

cd ../google-cloud-java
cd "../${merged_repository}"
git remote add ${service} ../${service}
git config --add secrets.allowed "dest.*src"
git fetch ${service} #--tags
Expand All @@ -60,7 +60,7 @@ do
rm -rf ../${service}
done

# cwd: monorepo/google-cloud-java
# cwd: monorepo/google-cloud-java-merged
echo "Working directory: $(pwd)"

cp -R ../../../google-cloud-jar-parent google-cloud-jar-parent
Expand All @@ -72,52 +72,58 @@ git commit -m 'chore: add template files'
../../generate_root_pom.sh

git add pom.xml
git commit -am 'chore: create aggregator pom'
git commit -am 'chore: create aggregator pom' --allow-empty

# Point modules poms and BOMs to the aggregator pom as parent
bash ../../set_parent_pom.sh

git add --all
git commit -am 'chore: point modules to the aggregator pom as parent'
git commit -am 'chore: point modules to the aggregator pom as parent' \
--allow-empty

../../consolidate_config.sh

git add --all
git commit -am 'chore: consolidate config to parent'
git commit -am 'chore: consolidate config to parent' \
--allow-empty

../../generate_gapic_bom.sh

# add the gapic bom module to root pom.xml by regenerating aggregator pom
../../generate_root_pom.sh

git add gapic-libraries-bom/pom.xml
git commit -am 'chore: create gapic-libraries-bom'
git commit -am 'chore: create gapic-libraries-bom' \
--allow-empty

cp ../../gapic_bom_versions.txt gapic-libraries-bom/versions.txt

../../delete_non_generated_samples.sh

git add --all
git commit -am 'chore: delete non-auto-generated samples'
git commit -am 'chore: delete non-auto-generated samples' \
--allow-empty

../../generate_root_versions_txt.sh
../../update_versions.sh -s
../../apply_current_versions.sh

git add --all
git commit -am 'chore: update versions to latest in maven'
git commit -am 'chore: update versions to latest in maven' \
--allow-empty

../../update_owlbot_postprocessor_config.sh

git add --all
git commit -am 'chore: remove and disable owlbot postprocessor templates'
git commit -am 'chore: remove and disable owlbot postprocessor templates' \
--allow-empty

for F in `find . -maxdepth 2 -name '.OwlBot.yaml'`; do sh ../../set_owlbot_config.sh $F; done
git commit -am 'chore: set owlbot copy config' --allow-empty

# create a monorepo/diff repo
cd ..
cp -R google-cloud-java split
cp -R "${merged_repository}" split
rm -rf split/.git
git clone -b main --single-branch https://github.com/googleapis/google-cloud-java.git shadow
cp -R shadow/.git split/.git
Expand Down
133 changes: 1 addition & 132 deletions generation/repos.txt
Original file line number Diff line number Diff line change
@@ -1,132 +1 @@
java-accessapproval
java-accesscontextmanager
java-aiplatform
java-analytics-admin
java-analytics-data
java-api-gateway
java-apigee-connect
java-apigee-registry
java-apikeys
java-appengine-admin
java-area120-tables
java-artifact-registry
java-asset
java-assured-workloads
java-automl
java-bare-metal-solution
java-batch
java-beyondcorp-appconnections
java-beyondcorp-appconnectors
java-beyondcorp-appgateways
java-beyondcorp-clientconnectorservices
java-beyondcorp-clientgateways
java-bigquery-data-exchange
java-bigqueryconnection
java-bigquerydatatransfer
java-bigquerymigration
java-bigqueryreservation
java-billing
java-billingbudgets
java-binary-authorization
java-certificate-manager
java-channel
java-cloudbuild
java-cloudcommerceconsumerprocurement
java-compute
java-contact-center-insights
java-container
java-containeranalysis
java-data-fusion
java-datacatalog
java-dataflow
java-dataform
java-datalabeling
java-dataplex
java-dataproc
java-dataproc-metastore
java-datastream
java-debugger-client
java-deploy
java-dialogflow
java-dialogflow-cx
java-dlp
java-dms
java-dns
java-document-ai
java-domains
java-errorreporting
java-essential-contacts
java-eventarc
java-eventarc-publishing
java-filestore
java-functions
java-game-servers
java-gke-backup
java-gke-connect-gateway
java-gke-multi-cloud
java-gkehub
java-grafeas
java-gsuite-addons
java-iam-admin
java-iamcredentials
java-ids
java-iot
java-kms
java-language
java-life-sciences
java-managed-identities
java-mediatranslation
java-memcache
java-monitoring
java-monitoring-dashboards
java-network-management
java-network-security
java-networkconnectivity
java-notebooks
java-notification
java-optimization
java-orchestration-airflow
java-orgpolicy
java-os-config
java-os-login
java-phishingprotection
java-policy-troubleshooter
java-private-catalog
java-profiler
java-recaptchaenterprise
java-recommendations-ai
java-recommender
java-redis
java-resource-settings
java-resourcemanager
java-retail
java-run
java-scheduler
java-secretmanager
java-security-private-ca
java-securitycenter
java-securitycenter-settings
java-service-control
java-service-management
java-service-usage
java-servicedirectory
java-shell
java-speech
java-storage-transfer
java-talent
java-tasks
java-texttospeech
java-tpu
java-trace
java-translate
java-video-intelligence
java-video-live-stream
java-video-stitcher
java-video-transcoder
java-vision
java-vmmigration
java-vpcaccess
java-webrisk
java-websecurityscanner
java-workflow-executions
java-workflows
java-logging