Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add release automation #416

Merged
merged 2 commits into from
Dec 2, 2022
Merged

Add release automation #416

merged 2 commits into from
Dec 2, 2022

Conversation

jhkennedy
Copy link
Collaborator

@jhkennedy jhkennedy commented Dec 1, 2022

This adds a collection of GitHub Actions intended to make releasing as simple as merging to main.

The release notes will be extracted from the CHANGELOG.md file and published as part of the release. The assumes:

  1. you update the CHANGELOG.md before releasing²
  2. you label the release PR with the level of changes following semantic versioning conventions

A terse description of the flow is:

  1. A PR is opened to main
    • changelog.yml will ensure the CHANGELOG.md file has been updated
    • labeled-pr.yml will ensure the PR is labeled with a label indicating the level of changes
      (what part of the semantic version tag to bump/increment)
  2. Once merged, tag.yml will compute the new version tag from the PR label and push a version tag to the repo
  3. When a new version tag is pushed, release.yml will find the appropriate release notes (changes) in CHANGELOG.md based on the version tag, and publish the release on GitHub.

Footnotes

  1. I've copied the release notes from release notes v0.2.0 #413, and all previous releases, into the CHANGELOG.md file.
  2. I've set the changelog.yml workflow to ensure every PR includes a changelog. After a release, developers should make a new section for the next release with a proposed version number indicating the level of changes included in their new feature. If the next release section already exists, they should add their changes to it and adjust the proposed number as needed. If there are no changes that should be communicated to users (e.g., CI/CD changes), the bumpless label can be used to say, "these are changes which do not require a version bump and do not need to be recorded in the changelog," and the check will pass.

TODOs:

  • Add the RELEASE_ACCESS_TOKEN repository secret
  • Add and stylize major, minor, patch, bumpless, and bot labels
  • write a release-process.md document describing the release process moved to Document the the release process #418

Screenshots (if appropriate):

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

  • I have added an explanation of what your changes do and why you'd like us to include them.
  • I have written new tests for your core changes, as applicable.
  • I have successfully ran tests with your changes locally.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly. Punted to Document the the release process #418

@jhkennedy
Copy link
Collaborator Author

Note: These are the same workflows used in https://github.com/ACCESS-Cloud-Based-InSAR/DockerizedTopsApp

Comment on lines +12 to +13
user: dbekaert
email: [email protected]
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dbekaert for ACCESS19, we setup a "bot" user which creates the release tag, releases, and syncs main back to dev if a release branch was used.

Here I've put you as this repo is under your personal GH account. That means it'll look like you did the tagging/releasing/syncing yourself. For this all to work, you'll need to create a personal access token with write access to this repo and add it as a secret named RELEASE_ACCESS_TOKEN to this repo.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok i will do it in the morning.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would create a jpl org email e.g. [email protected] and have you, Jeremy, and Joe be the admins on this mailing list. This can then function as your bot - only need to create a github account for this email.

@jhkennedy
Copy link
Collaborator Author

I've moved

write a release-process.md document describing the release process

to #418 -- this doesn't need to block the release but should be done soon.

@jhkennedy jhkennedy marked this pull request as ready for review December 1, 2022 20:02
@dbekaert dbekaert mentioned this pull request Dec 1, 2022
@dbekaert
Copy link
Owner

dbekaert commented Dec 1, 2022

@jhkennedy i added in the release access token. Can you confirm we are good? feel free to merge if all good.

Copy link
Owner

@dbekaert dbekaert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks @jhkennedy

fix conventional to not be supported
@jhkennedy
Copy link
Collaborator Author

@jhkennedy i added in the release access token. Can you confirm we are good? feel free to merge if all good.

@dbekaert I cannot. I don't have access to any repository settings, and I think only you do. If you screenshot the secrets page, I could confirm. But it's CI/CD so "ship it to try it" 🤣

@jlmaurer jlmaurer merged commit 38302be into dbekaert:dev Dec 2, 2022
@jhkennedy jhkennedy deleted the release-flow branch December 2, 2022 17:53
jlmaurer added a commit that referenced this pull request Dec 3, 2022
* fix typo when setting model level

* fix an import bug and clean up unit test_scenario_1

* fix a couple import bugs

* update files with new accurate pressure model delays

* fix a pressure level specification bug in era5

* Update files

* add in ecef2enu

* start some new unit tests for test_util

* comment out extra LOS calcs (From LOS file)

* Add in ecef2enu

* convert ECEF unit vectors to ENU

* Fix LOS calculation and remove print statements

* add a slant range threshold for sanity checking los

* Ensure that the geo2rdr object is deleted

* update unit tests and fix a bug

* Bunch of clean up and making everything consistent

* fix a couple bugs

* Add unit tests and docstrings

* refactor losreader to be cleaner, add unit tests

* clarify a couple points in losreader

* start refactoring LOS calc

* move some files around

* BB_few_fixes

change name of variable (los_file -> lookvector)
fix masking with a reshape
make shape of outputs consistent (LOS file, orbit file)

* Ensure that input shape is preserved in losreader

* fix and write

* remove extra write

* test fixes from Jeremy

* Fix call to getLookVectors

* Fix some bugs and make sure los vars get carried through correctly

* Fix unit tests for GMAO and a few bugs

* autopep8 fixes

* add in dummy file; github does not like using dummy.txt

* update unit test

* use dem_stitcher

* add new line add end of environment.yml

* remove extraneous code

* update env-dev

* put new dependencies into pip

* add geopandas

* remove extra functions and update docstring of getDEM

* remove DEM download from test suite

* Add a mamba v. 2 (#323)

* update dependencies

* fix a typo

* Take care of GNSS duplicates

* Fix how dups are dropped and add a unit test for it

* remove unneeded test breaking

* add fixed zlevels to other weather models ie non ECMWF

* identify tests that are long

* Specify tests as long and skip on circleci

* skip tests that require file download if they do not exist

* skip tests that require file download if they do not exist

* Fix bugs and remove old breakpoint statements

* Pass closest localtime as opposed to mean within range

* fix a few typos

* Fix a few more bugs, and finish off the conventional calculation

* updated WeatherModes.md with revised information

* fix import statements in the unit test suite

* Fix local-time bug

* Remove reference to local env

* fix a bug for parsing los

* Fix a bug when reading lat/lon files

* Fix bug, was expecting localtime

The code had expected a localtime key, which led to crashes in cases where a user does not wish to perform localtime analyses

* remove breakpoint from gmao.py

* update dem.py with new dem-stitcher api

* update getDEM to use the new dem-stitcher API

* Fix dem-stitcher version for the current API

* Add dev dep "autopep8"

The PR template suggests running it on your changes.

* autopep8

* flake8

* flake8

* flake8

* install via conda instead of pip

* Remove unused writeWeatherVars2HDF5()

* flake8

* Install via conda instead of pip

* Raise better error for missing subdatasets

* flake8

Somehow these changes flew under the radar. Yes, I am aware of the irony of that statement.

* move all pip deps except dem-stitcher to conda deps

* Update dev env file to reflect dem-stitcher update

* Update env file to reflect dem-stitcher update

* Update dev env file to newest dem-stitcher

* Update env file to newest dem-stitcher

* stash dep changes

* update environment deps

* correct name of dem_stitcher conda package

* remove uneeded deps

* Update ERA5 API

* autopep8

* Version 0.1.0

* Update readme, clean up old files (#360)

* Initial readme update

* clean up old files,readme

* fix contrib typos

Co-authored-by: Jeremy Maurer <[email protected]>

* Adding env file for using isce3

* Remove use of geometry extension

* Pulling in #332 and updating tests

* Pull Earthdata from netrc

* Using UTC time tags and removing ref time for state vectors

* Use instance to check for zenith - working zenith tests

* Fixing typos. Working LOS tests

* Using isce3 for look angle computation - Conventional

* Fix for unit tests

* Making isce3 the default conda envs

* Controlling import of isce3 to circumvent pycoverage's double import

* Trying CI with isce3 pinned

* Quick bump up of python version

* change to config file format

* few tweaks to cfg file

* autopep8 suggestions

* Adding ECEF look vector computation

* Adding Top of Atmosphere calculation

* Adding simple unit test for raytracing

* updates for new yaml file config

* Refactoring getTopOfAtmosphere

* Refactoring to use bounds only always to load models

* Fix for unit tests for llreader

* Fix bug in managing GDAL config option

* update raider config yaml

* Completely replace gdal with rasterio + pyproj

* Changes based on feedback

* Use rasterio profile everywhere

* PEP8 cleanup

* fix spacing

* update checkArgs to work with yaml input

* Carry through config file changes to dem.py

* Convert to entrypoints instead of scripts for better portability

* intiial switch to pyproject.toml

* drop hashbangs

* clean up setup.py

* tweak circleci

* really, really remove environemnt-dev.yml

* Add tests for entrypoints

* drop redundant entrypoint tests from circle ci

* Cube generation prototype

* Feedback from review

* remove cartopy as its causing an issue and fix a few typos

* fix fwe typos

* Transition scripts to entry points

Added a main to every one of the modules we will be using for an entry point and used said main function for the entry points in the setup.py.

* ZTD for bounding box is workinggit status

* fix typo

* update test suite with new APIs and temporarily disable IO tests

* fix bug in test suite

* remove depracated tests

* Ray tracing cube prototype

* Getting golden tests to work with new parser

* Checking in cube generation

* Removing prepareCube

* Fixes for cubes in non lat/lon systems

* Minor optimization of ray tracing to speed up computation

* Documenting ray intersection approach

* Add dockerfile

* Use git object name for container tag

* Add authors and drop vendor from docker image labels

* add dependabot to keep actions up to date

* Fix hrrr (#379)

* update default yaml

* initial try at updating hrrr

* initial working HRRR

* need to keep projection info for HRRR

* add herbie to environment.yml

* Update environment.yml

Co-authored-by: Jeremy Maurer <[email protected]>

* Update Dockerfile

Co-authored-by: dbekaert <[email protected]>

* remove a few typos

* Fix some typos, add a _time_res attribute to weather models

* Fix some bugs and get basic ztd working

* add geocoded_file and move all cube calcs to cube function

* fix file naming bug that was keeping delay cube from being written

* fix but in test suite

* update unit tests

* fix bug in drop_nans

* fix nan-dropping bug

* fix gmao api call

* add back runtime parameter parsing

* Update build.yml

* Prepping for chunking with multiprocessing

* Making output arrays optional

* Multiprocessing work in progress

* Fixes for handling model info

* Updated HRRR subsetter

* Update packaging to use pyproj and generate a dynamic version number (#385)

* convert most project metadata to pyproject.toml

* add setuptools_scm to environment

* drop version.txt

* Add __version__ to package

* force conda-forge

* tag docker with python version too

* Update pyproject.toml

Co-authored-by: dbekaert <[email protected]>

* Fix typo

Co-authored-by: dbekaert <[email protected]>

* Updating HRRR with subsetting and correct conventions

* Adding support to write netcdf in other projections

* Minor cleanup

* Write support for HRRR

* Working HRRR Zenith Delays

* Working ray tracing for HRRR

* Addressing feedback

* automatically extend buffer when raytracing

* clarify between conventional std and raytracing

* Need to return a list of arrays with multi-band rasters

* add variable to context manager so hgts are retrievied

* updates needed for RAiDER_tutorial notebook (#394)

* add jupyter to conda install

* add wand for pdf viewing in jupyter

* check for heights in actual dataframe

* fix bug in writing delays for gnss station file

* correct output filename for stations

* automatically change file extension to nc if not nc or h5

* force rasterio version to 3.3

* make rasterio >= 1.3.3

* begin support for cli interface to dlGNSS and specific steps for delay (#401)

* begin support for cli interface to dlGNSS and specific steps for delay

* doStep is now supported and enables GNSS download

* doStep now fully works

* download GNSS stations works

* change name of gps output dir

* fix HRRR z-level sign issue by using native levels (#406)

Co-authored-by: Jeremy Maurer <[email protected]>

* support reading the AOI options (#402)

* support reading lat/lons from a GUNW

* enable calculation of delay with a GUNW file and specified height levels

* extra check to make sure GNSS file is treated separately

* update test to reflect change in output of rio_extent (from WSEN to SNWE)

* match type for updated test

* enable reading of bbox from ISCE geocoded lat/lon files

* calc delays at DEM heights

* remove accidentally addited GNSS cli

* Projection bounds (#409)

* validator bug fix

* support for reusing weather model file in projected coords (HRRR)

* remove an if condition and extra comments

* Close to finishing off ztd release (#411)

* make sure the right steps happens and default to only weather model calculation

* Add dem download if hgt file not defined for latlon inputs

* fix a bounding box bug for GMAO

* fix bounding box problem and remove broken models

* Update README.md

Does not need the env to create it the first time.

* Update README.md

Adding Piyush to the README as contributor

* Fix some parsing and extent checking bugs

* Update processWM.py

update log message

* Update runProgram.py

fix for showing debug messages correctly

* Update logger.py

correctly show debug messages when user requests verbose

Co-authored-by: Jeremy Maurer <[email protected]>
Co-authored-by: dbekaert <[email protected]>
Co-authored-by: BB <[email protected]>

* change CLI to raider.py and fns to main (#414)

* change CLI to raider.py and fns to main

* fix fn name in tests

* update circleci; start fixing old functions in downloadGNSS

* replace raiderDelay with raider everywhere

* downloadGNSS notebook and CLI (dostep) works

* relocate downloadGNSS and make it work

* bugfix for reading weather files

* remove steps functionality

* add option for only downloading weather model

* put back files git deleted

* Update README.md

* Update README.md (#415)

* Update README.md

Fix the TOC and hyperlinking.

* describe pulling specific releases (#417)

* Fix CircleCI buid and cleanup config (#422)

* temporary gdal fix

* cleanup circleci config and switch to micromamba

* ensure micromamba is available

* constrain isce3 build

* try revert gdal pin

* Constrain python version for version info workflow

* try * in buildstring

Co-authored-by: Jeremy Maurer <[email protected]>

* Update unit tests (#419)

* update unit tests

* update unit tests and add end-to-end tests

* make end-to-end tests long

* add the orbit file

* take out conventional slant for now

* golden unit tests

* add golden datasets

* update golden datasets paths

* move files

Co-authored-by: Jeremy Maurer <[email protected]>

* Add release automation (#416)

* Add release automation

* Update CHANGELOG.md

fix conventional to not be supported

Co-authored-by: dbekaert <[email protected]>

* remove long test flags (#423)

* remove long test flags

* Skip CDSAPI test for now, passing locally but failing for CircleCI

Co-authored-by: Jeremy Maurer <[email protected]>

Co-authored-by: bbuzz31 <[email protected]>
Co-authored-by: cmarshak <[email protected]>
Co-authored-by: sssangha <[email protected]>
Co-authored-by: Eric Jameson Fielding <[email protected]>
Co-authored-by: Simran S Sangha <[email protected]>
Co-authored-by: Scott Staniewicz <[email protected]>
Co-authored-by: Jeremy Maurer <[email protected]>
Co-authored-by: Jeremy Maurer <[email protected]>
Co-authored-by: the-garlic-os <[email protected]>
Co-authored-by: piyushrpt <[email protected]>
Co-authored-by: Joseph H Kennedy <[email protected]>
Co-authored-by: MLicari1 <[email protected]>
Co-authored-by: dbekaert <[email protected]>
@dbekaert dbekaert mentioned this pull request Dec 5, 2022
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants