Setup automated releases from GitHub Actions CI #867

olafurpg · 2020-09-18T08:57:47Z

Previously, releases were done from a local computer with the
sbt-release plugin. Now, releases happen automatically from GitHub
Actions. Every merge into master should trigger a SNAPSHOT release
and every git tag push should trigger a stable Sonatype release.

For more details, see https://github.com/olafurpg/sbt-ci-release

regadas

This is super nice! 🎉 I guess this just speeds up the process of moving completely to GHA as we did in chill.

Now @posco just needs to add a few secrets to the repo! 🚀 (I don't have enough perms)

olafurpg · 2020-09-18T09:04:55Z

I just received admin access for this repo and configured the Sonatype+PGP secrets. It may take several attempts after this is merged to validate that everything is working as expected.

I guess this just speeds up the process of moving completely to GHA as we did in chill.

Yes, I think it would be a nice followup PR to move all the CI to GitHub Actions.

olafurpg · 2020-09-18T09:05:31Z

I'm planning to followup with similar changes in the chill and bijection repos.

Previously, releases were done from a local computer with the sbt-release plugin. Now, releases happen automatically from GitHub Actions. Every merge into master should trigger a SNAPSHOT release and every git tag push should trigger a stable Sonatype release. For more details, see htts://github.com/olafurpg/sbt-ci-release

codecov-commenter · 2020-09-18T09:06:53Z

Codecov Report

Merging #867 into develop will decrease coverage by 0.04%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           develop     #867      +/-   ##
===========================================
- Coverage    89.32%   89.27%   -0.05%     
===========================================
  Files          117      117              
  Lines         9326     9326              
  Branches       365      365              
===========================================
- Hits          8330     8326       -4     
- Misses         996     1000       +4

Impacted Files	Coverage Δ
...in/scala/com/twitter/algebird/scalacheck/Gen.scala	`95.34% <0.00%> (-4.66%)`	⬇️
.../main/scala/com/twitter/algebird/Successible.scala	`87.50% <0.00%> (-4.17%)`	⬇️
...src/main/scala/com/twitter/algebird/Interval.scala	`80.00% <0.00%> (-0.87%)`	⬇️
.../main/scala/com/twitter/algebird/BloomFilter.scala	`94.24% <0.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fdd3aba...2d33301. Read the comment docs.

olafurpg · 2020-09-18T09:10:01Z

version.sbt

@@ -1 +0,0 @@
-version in ThisBuild := "0.13.8-SNAPSHOT"


This changes the local value of version to be formatted like this 0.13.7+34-2d333018-SNAPSHOT. Is that an acceptable change? I think it's possible to get configure the version to not include the git describe information you prefer to keep the old behavior.

imo i think it's ok.

regadas · 2020-09-18T09:13:27Z

Yeah following up with a PR would be nice... I've been meaning to do this for a long time lol

regadas · 2020-09-18T10:22:40Z

.github/workflows/release.yml

+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v1
+      - uses: olafurpg/setup-scala@v7


One i thing i realised is that we can add some artifact cache here

- name: cache SBT uses: coursier/cache-action@v3

This job only runs the releases, which I think is desirable to run with a clean machine. My experience is that GitHub Actions is usually fast enough for most projects without caching

right, anecdotally I've seen it fail a few times ¯_(ツ)_/¯ I guess we can always re-trigger?

You can always rerun the job or force push the git tag

right that's what I meant. I understand that this is probably a preference thing but i don't feel that fetching artifacts every time offer a lot of guarantees. If a build is green on develop for a given hash, running the release job (develop or tag) should mean that it will equally build and not be subjective to transient errors. Publishing to sonatype should be the only thing that can fail (which fails quite often unfortunately)

johnynek

This is great and super exciting!

One thing we probably want to address: the CI is a bit flaky on two kinds of tests: 1. approximate laws related to floating point numbers, 2. tests checking probabilistic bounds (this is true with Prob > y, which we run many trials and see if the result is highly improbable given that).

We should probably tune it so that those rarely happen.

olafurpg · 2020-09-22T10:58:53Z

Merging to see what happens on master. Small heads up, it usually takes a few failed attempts before it works correctly

olafurpg · 2020-09-22T11:53:26Z

The SNAPSHOT release worked, you can download the artifacts with

cs fetch com.twitter:algebird-core_2.11:0.13.7+37-fd6454ad-SNAPSHOT -r sonatype:snapshots

The next step is to try a stable release by pushing a git tag. Any objections if I push a v0.13.8-M1 tag to see if everything works?

FWIW, I have a git-release script in my $PATH

❯ cat `which git-release`
set -eux
version=$1
git tag -af "v$version" -m "v$version" && git push -f origin v$version

I normally cut releases by running git release 0.13.8 from the terminal. I usually wait until the CI release job has completed before publishing a release on GitHub.

olafurpg · 2020-09-22T11:56:35Z

One thing we probably want to address: the CI is a bit flaky on two kinds of tests: 1. approximate laws related to floating point numbers, 2. tests checking probabilistic bounds (this is true with Prob > y, which we run many trials and see if the result is highly improbable given that).

The CI release currently runs the publish step without testing so flaky failures should not block the release. It's possible to add custom steps before +publishSigned in case you prefer that, but when I tried this in the past I normally ended up spending more time dealing with incomplete releases than catching actual bugs before a release (the master branch has presumably gone through PR code review).

regadas · 2020-09-22T12:08:47Z

v0.13.8-M1

we can go with a v0.13.8 since there are dependency updates, #844, #829 and some other minor things.

It's possible to add custom steps before +publishSigned

Yeah we can always just add sbt +test if needed but +1 on leaving it as is for now.

regadas · 2020-09-22T12:11:44Z

Let's include #850 as well

regadas · 2020-09-23T15:01:38Z

I think we are good for a new release v0.13.8.

who should do the honours? I can prepare the release notes if you want /cc @johnynek @olafurpg

johnynek · 2020-09-23T19:01:10Z

One argument is the honors should go to @olafurpg who once again did the important work of making releases easier, and helping remove friction from the ecosystem. My second nomination would be @regadas who has really stepped up to do a lot of work managing these libraries. For the first time you will be able to release since you don't need the twitter sonatype credentials.

I'm happy to do it as well... :)

olafurpg · 2020-09-25T08:58:44Z

I pushed a tag for v0.13.8-M1 and it seems to have worked

❯ cs resolve com.twitter:algebird-core_2.12:0.13.8-M1
com.googlecode.javaewah:JavaEWAH:1.1.7:default
com.twitter:algebird-core_2.12:0.13.8-M1:default
org.scala-lang:scala-library:2.12.12:default
org.scala-lang:scala-reflect:2.12.12:default
org.scala-lang.modules:scala-collection-compat_2.12:2.2.0:default
org.typelevel:algebra_2.12:2.0.0:default
org.typelevel:cats-kernel_2.12:2.0.0:default

I opened #871 to make it easier to write release notes. I'll follow up with similar changes in the chill and bijection repos, although it may take a few days until I get around to it.

regadas · 2020-09-25T14:19:02Z

it worked! 🎉 however i thought you were releasing v0.13.8

regadas · 2020-09-25T14:21:45Z

@johnynek thx for the acknowledgement 🙏

olafurpg · 2020-09-26T08:45:03Z

You can do the non-milestone release, I just wanted to validate the pipeline was working :)

Follow up to twitter/algebird#867 Previously, releases were done from a local computer with the sbt-release plugin. Now, releases happen automatically from GitHub Actions. Every merge into master should trigger a SNAPSHOT release and every git tag push should trigger a stable Sonatype release. For more details, see olafurpg/sbt-ci-release

olafurpg · 2020-12-02T09:38:20Z

Followup PRs

Follow up to twitter/algebird#867 Previously, releases were done from a local computer with the sbt-release plugin. Now, releases happen automatically from GitHub Actions. Every merge into master should trigger a SNAPSHOT release and every git tag push should trigger a stable Sonatype release. For more details, see olafurpg/sbt-ci-release

olafurpg requested a review from johnynek September 18, 2020 08:58

regadas approved these changes Sep 18, 2020

View reviewed changes

olafurpg force-pushed the ci-release branch from c32e889 to 2d33301 Compare September 18, 2020 09:06

olafurpg commented Sep 18, 2020

View reviewed changes

regadas reviewed Sep 18, 2020

View reviewed changes

johnynek approved these changes Sep 18, 2020

View reviewed changes

olafurpg merged commit fd6454a into twitter:develop Sep 22, 2020

olafurpg deleted the ci-release branch September 22, 2020 10:58

olafurpg mentioned this pull request Dec 2, 2020

Setup automated releases from GitHub Actions CI twitter/chill#507

Merged

olafurpg mentioned this pull request Dec 2, 2020

Setup automated releases from GitHub Actions CI twitter/bijection#460

Merged

johnynek mentioned this pull request Mar 11, 2022

Set up automatic publishing twitter/scalding#1973

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Setup automated releases from GitHub Actions CI #867

Setup automated releases from GitHub Actions CI #867

olafurpg commented Sep 18, 2020 •

edited

Loading

regadas left a comment •

edited

Loading

olafurpg commented Sep 18, 2020

olafurpg commented Sep 18, 2020

codecov-commenter commented Sep 18, 2020 •

edited

Loading

olafurpg Sep 18, 2020

regadas Sep 18, 2020

regadas commented Sep 18, 2020

regadas Sep 18, 2020

olafurpg Sep 18, 2020

regadas Sep 18, 2020

olafurpg Sep 18, 2020

regadas Sep 18, 2020

johnynek left a comment

olafurpg commented Sep 22, 2020

olafurpg commented Sep 22, 2020

olafurpg commented Sep 22, 2020

regadas commented Sep 22, 2020 •

edited

Loading

regadas commented Sep 22, 2020

regadas commented Sep 23, 2020

johnynek commented Sep 23, 2020

olafurpg commented Sep 25, 2020

regadas commented Sep 25, 2020

regadas commented Sep 25, 2020

olafurpg commented Sep 26, 2020

olafurpg commented Dec 2, 2020

Setup automated releases from GitHub Actions CI #867

Setup automated releases from GitHub Actions CI #867

Conversation

olafurpg commented Sep 18, 2020 • edited Loading

regadas left a comment • edited Loading

Choose a reason for hiding this comment

olafurpg commented Sep 18, 2020

olafurpg commented Sep 18, 2020

codecov-commenter commented Sep 18, 2020 • edited Loading

Codecov Report

olafurpg Sep 18, 2020

Choose a reason for hiding this comment

regadas Sep 18, 2020

Choose a reason for hiding this comment

regadas commented Sep 18, 2020

regadas Sep 18, 2020

Choose a reason for hiding this comment

olafurpg Sep 18, 2020

Choose a reason for hiding this comment

regadas Sep 18, 2020

Choose a reason for hiding this comment

olafurpg Sep 18, 2020

Choose a reason for hiding this comment

regadas Sep 18, 2020

Choose a reason for hiding this comment

johnynek left a comment

Choose a reason for hiding this comment

olafurpg commented Sep 22, 2020

olafurpg commented Sep 22, 2020

olafurpg commented Sep 22, 2020

regadas commented Sep 22, 2020 • edited Loading

regadas commented Sep 22, 2020

regadas commented Sep 23, 2020

johnynek commented Sep 23, 2020

olafurpg commented Sep 25, 2020

regadas commented Sep 25, 2020

regadas commented Sep 25, 2020

olafurpg commented Sep 26, 2020

olafurpg commented Dec 2, 2020

olafurpg commented Sep 18, 2020 •

edited

Loading

regadas left a comment •

edited

Loading

codecov-commenter commented Sep 18, 2020 •

edited

Loading

regadas commented Sep 22, 2020 •

edited

Loading