Separate dev and production deployments in GitHub workflow #382
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…n into one workflow
…er build and push
dwinston
reviewed
Nov 16, 2023
| branches: | ||
| - main | ||
| tags: | ||
| - 'v*' |
There was a problem hiding this comment.
ooh, I didn't know about this wildcard syntax. cool.
dwinston
requested changes
Nov 16, 2023
dwinston
pushed a commit
that referenced
this pull request
Nov 20, 2023
* Consolidate workflows for building docker images and deploying to Spin into one workflow * Remove docker-build.sh in favor of letting GitHub Actions handle Docker build and push * Update Release Process doc with info about initiating via GitHub Releases * Replace Rancher-Action with generic HTTP call * Replace release event with tag push event, which is required for semver metadata * Remove unnecessary pr event * Add more release instructions
dwinston
added a commit
that referenced
this pull request
Nov 20, 2023
* fix: Handle `anyOf` in JSON Schema property * Indicate function return value type * Refactor function and add comments * fix: Prefix class name with `nmdc:` * Implement helper function to process both single-ref and multi-ref specs * Document prefix functionality * Fix punctuation in comment * Update dictionary and function to accommodate multiple classes per collection * WIP: Update doc link maker to accommodate collections that map to multiple classes * Clarify variable names * Add comments in an attempt to clarify code * Delete commented-out code that doesn't accommodate multi-class collections * Add tests covering some corner cases * Fix inaccurate type hint * Clarify docstring * Replace reference to nonexistent dict and implement preliminary patch * Make the collection name bold on the search page * Update search page to account for collections mapping to multiple classes * Remove redundant type hints * style: black format * panic on no-type given * add script and api function * update script * Refactor runtime client methods to raise for status and parse and return results * handle omics processing records * update docstring * update to include correct prefix * update to use use new insdc_bioproject_identifiers slot on omics_processing * style: black format * add typecodes enpoint (#386) unauthenticated. closes #385 * update .gitpod.yml * add sshproxy.sh for nersc tunneling * update make cmd * add gitpod affordance * add gitpod dockerfile * update gitpod stuff * rename * update Makefile * fix * gitpod: pull dev mdb * fix * fix make target * Separate dev and production deployments in GitHub workflow (#382) * Consolidate workflows for building docker images and deploying to Spin into one workflow * Remove docker-build.sh in favor of letting GitHub Actions handle Docker build and push * Update Release Process doc with info about initiating via GitHub Releases * Replace Rancher-Action with generic HTTP call * Replace release event with tag push event, which is required for semver metadata * Remove unnecessary pr event * Add more release instructions * style: fix, and elaborate a bit * Revert stuff This reverts commit 1b2372d. * style: fix, and elaborate a bit --------- Co-authored-by: eecavanna <[email protected]> Co-authored-by: Donny Winston <[email protected]> Co-authored-by: Michael Thornton <[email protected]> Co-authored-by: Donny Winston <[email protected]> Co-authored-by: Jing Cao <[email protected]> Co-authored-by: Patrick Kalita <[email protected]>
dwinston
pushed a commit
that referenced
this pull request
Nov 28, 2023
* Consolidate workflows for building docker images and deploying to Spin into one workflow * Remove docker-build.sh in favor of letting GitHub Actions handle Docker build and push * Update Release Process doc with info about initiating via GitHub Releases * Replace Rancher-Action with generic HTTP call * Replace release event with tag push event, which is required for semver metadata * Remove unnecessary pr event * Add more release instructions
dwinston
added a commit
that referenced
this pull request
Nov 28, 2023
* fix: Handle `anyOf` in JSON Schema property * Indicate function return value type * Refactor function and add comments * fix: Prefix class name with `nmdc:` * Implement helper function to process both single-ref and multi-ref specs * Document prefix functionality * Fix punctuation in comment * Update dictionary and function to accommodate multiple classes per collection * WIP: Update doc link maker to accommodate collections that map to multiple classes * Clarify variable names * Add comments in an attempt to clarify code * Delete commented-out code that doesn't accommodate multi-class collections * Add tests covering some corner cases * Fix inaccurate type hint * Clarify docstring * Replace reference to nonexistent dict and implement preliminary patch * Make the collection name bold on the search page * Update search page to account for collections mapping to multiple classes * Remove redundant type hints * style: black format * panic on no-type given * add script and api function * update script * Refactor runtime client methods to raise for status and parse and return results * handle omics processing records * update docstring * update to include correct prefix * update to use use new insdc_bioproject_identifiers slot on omics_processing * style: black format * add typecodes enpoint (#386) unauthenticated. closes #385 * update .gitpod.yml * add sshproxy.sh for nersc tunneling * update make cmd * add gitpod affordance * add gitpod dockerfile * update gitpod stuff * rename * update Makefile * fix * gitpod: pull dev mdb * fix * fix make target * Separate dev and production deployments in GitHub workflow (#382) * Consolidate workflows for building docker images and deploying to Spin into one workflow * Remove docker-build.sh in favor of letting GitHub Actions handle Docker build and push * Update Release Process doc with info about initiating via GitHub Releases * Replace Rancher-Action with generic HTTP call * Replace release event with tag push event, which is required for semver metadata * Remove unnecessary pr event * Add more release instructions * style: fix, and elaborate a bit * Revert stuff This reverts commit 1b2372d. * style: fix, and elaborate a bit --------- Co-authored-by: eecavanna <[email protected]> Co-authored-by: Donny Winston <[email protected]> Co-authored-by: Michael Thornton <[email protected]> Co-authored-by: Donny Winston <[email protected]> Co-authored-by: Jing Cao <[email protected]> Co-authored-by: Patrick Kalita <[email protected]>
dwinston
added a commit
that referenced
this pull request
Nov 28, 2023
* add stuff * Update json.dump() to json.dumps() in user.py * Mix in orcid jwt flow with client_credentials flow closes #333 * update .gitpod.yml * add sshproxy.sh for nersc tunneling * update make cmd * add gitpod affordance * add gitpod dockerfile * update gitpod stuff * rename * update Makefile * fix * gitpod: pull dev mdb * fix * fix make target * Separate dev and production deployments in GitHub workflow (#382) * Consolidate workflows for building docker images and deploying to Spin into one workflow * Remove docker-build.sh in favor of letting GitHub Actions handle Docker build and push * Update Release Process doc with info about initiating via GitHub Releases * Replace Rancher-Action with generic HTTP call * Replace release event with tag push event, which is required for semver metadata * Remove unnecessary pr event * Add more release instructions * fix: Handle `anyOf` in JSON Schema property (#379) * fix: Handle `anyOf` in JSON Schema property * Indicate function return value type * Refactor function and add comments * fix: Prefix class name with `nmdc:` * Implement helper function to process both single-ref and multi-ref specs * Document prefix functionality * Fix punctuation in comment * Update dictionary and function to accommodate multiple classes per collection * WIP: Update doc link maker to accommodate collections that map to multiple classes * Clarify variable names * Add comments in an attempt to clarify code * Delete commented-out code that doesn't accommodate multi-class collections * Add tests covering some corner cases * Fix inaccurate type hint * Clarify docstring * Replace reference to nonexistent dict and implement preliminary patch * Make the collection name bold on the search page * Update search page to account for collections mapping to multiple classes * Remove redundant type hints * style: black format * panic on no-type given * add script and api function * update script * Refactor runtime client methods to raise for status and parse and return results * handle omics processing records * update docstring * update to include correct prefix * update to use use new insdc_bioproject_identifiers slot on omics_processing * style: black format * add typecodes enpoint (#386) unauthenticated. closes #385 * update .gitpod.yml * add sshproxy.sh for nersc tunneling * update make cmd * add gitpod affordance * add gitpod dockerfile * update gitpod stuff * rename * update Makefile * fix * gitpod: pull dev mdb * fix * fix make target * Separate dev and production deployments in GitHub workflow (#382) * Consolidate workflows for building docker images and deploying to Spin into one workflow * Remove docker-build.sh in favor of letting GitHub Actions handle Docker build and push * Update Release Process doc with info about initiating via GitHub Releases * Replace Rancher-Action with generic HTTP call * Replace release event with tag push event, which is required for semver metadata * Remove unnecessary pr event * Add more release instructions * style: fix, and elaborate a bit * Revert stuff This reverts commit 1b2372d. * style: fix, and elaborate a bit --------- Co-authored-by: eecavanna <[email protected]> Co-authored-by: Donny Winston <[email protected]> Co-authored-by: Michael Thornton <[email protected]> Co-authored-by: Donny Winston <[email protected]> Co-authored-by: Jing Cao <[email protected]> Co-authored-by: Patrick Kalita <[email protected]> * Update build-and-release-to-spin.yml * fix: Update build-and-release-to-spin.yml * Replace illegal variable defined using other variable (#389) Co-authored-by: Donny Winston <[email protected]> * Do full depth checkout so setuptools-scm can detect version * Rever to using env context for variables * Use local path context when running docker build so that it knows about git tags * Actually push the docker image, but not if running in a fork * Fix use of boolean var * Env context only available in `steps.if` * Update .dockerignore * Update main.py to display scm version * Allow study metadata not captured in submission portal to be passed to SubmissionPortalTranslator * Allow passing doi category to SubmissionPortalTranslator, use dataset_doi by default * Connect additional study translator parameters to Dagster op inputs * Allow additional Biosample metadata to be passed in via external CSV file * Add PyPI URL and elaborate on manual publishing process * Add related links section to `README.md` * ensure no w3id.org loop; use data portal API (#403) closes #402 * Mix in orcid jwt flow with client_credentials flow closes #333 * style: rm print statement * style: add docstring * style: add commentary * style: rm print-debugging * fix: rm unneeded import --------- Co-authored-by: Jing Cao <[email protected]> Co-authored-by: Patrick Kalita <[email protected]> Co-authored-by: eecavanna <[email protected]> Co-authored-by: eecavanna <[email protected]> Co-authored-by: Michael Thornton <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
These changes consolidate the old
build-and-push-docker-images.ymlandrelease-to-spin.ymlworkflows into one new one (build-and-release-to-spin.yml). The new workflow is responsible for building the two docker images, pushing them to Docker Hub with the correct tags, and redeploying Spin workloads. The Docker image tags are generated by thedocker/metadata-action.When the new workflow runs on a push to the
mainbranch, it will push Docker images with themaintag and then redeploy workloads in thenmdc-devnamespace. When the workflow runs on a push to a tag likevX.Y.Zit will push docker images with theX.Y.Ztag and theprodtag and then redeploy workloads in thenmdcnamespace. Rancher will need to be manually updated so that thenmdcnamespace workloads pull theprod-tagged image.To avoid confusion I completely removed the now unused
docker-build.shscript which tagged images with commit hashes. I lightly reformatted part of the Release Process doc page which was still referring to actions being triggered on changes toRELEASES.md(that hasn't been accurate for a while now). I intend to do more in-depth documentation with https://github.com/microbiomedata/infra-admin/issues/19.Fixes #357