From ef2d5e2d6111f3ec19b4c85c5dba404b94553621 Mon Sep 17 00:00:00 2001 From: Connor Clark Date: Wed, 18 Sep 2019 17:49:49 -0700 Subject: [PATCH 01/11] releasing --- docs/releasing.md | 103 ++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 95 insertions(+), 8 deletions(-) diff --git a/docs/releasing.md b/docs/releasing.md index e65cc3475f88..ab92db200a6a 100644 --- a/docs/releasing.md +++ b/docs/releasing.md @@ -4,7 +4,25 @@ ### Cadence -We ship once a month, on the Thursday before the 1st. While not necessary, followup minor/patch releases may be done if warranted. The planned ship dates are added to the internal Lighthouse calendar. +We aim to release every 3 weeks. Our schedule is set as follows: One day before the [expected Chromium branch point](https://www.chromium.org/developers/calendar) (which is every six weeks) and again exactly 3 weeks after that day. + +Example, the next fews releases (as of this writing) are: + +* Sep 24 2019 +* _Oct 15 2019_ +* Nov 5 2019 +* _Nov 26 2019_ +* Dec 17 2020 +* _Jan 7 2020_ +* Jan 28 2020 +* _Feb 18 2020_ +* Mar 10 2020 + +Underlined dates are the day before the expected Chromium branch point. + +The planned ship dates are added to the internal Lighthouse calendar. + +If a release is necessary outside these scheduled dates, we may choose to skip the next scheduled release. ### Release manager @@ -16,6 +34,8 @@ Release manager follows the below _Release Process_. ### Release publicity +Note: actively undergoing changes by @exterkamp and @egsweeny. + 1. Release mgr copies changelog to a new [Releases](https://github.com/GoogleChrome/lighthouse/releases). Tags and ships it. * Include a line of `We expect this release to ship in the DevTools of Chrome XX`. 1. Release mgr tells the _LH public_ Hangout chat about the new version. @@ -25,21 +45,84 @@ Release manager follows the below _Release Process_. ### Versioning -We follow [semver](https://semver.org/) versioning semantics (`vMajor.Minor.Patch`), to align with the greater Node community. Generally, this means our bi-weekly releases will bump a minor. Though we will release a new patch version if high-priority fixes are required before the next schedule release. Additionally, if a schedule release contains no new features, then we'll only bump the patch version. +We follow [semver](https://semver.org/) versioning semantics (`vMajor.Minor.Patch`). Breaking changes will bump the major version. New features or bug fixes will bump the minor version. If a release contains no new features, then we'll only bump the patch version. ## Release Process +Note: You'll wanna be on a Linux machine, since the Lightrider step will require that. + +### On the scheduled release date + +Before starting, you should announce to the LH eng channel that you are releasing, +and that no new PRs should be merged until you are done. If you have to go through all these steps +again to assert that release-blocking fixes work, other PRs may be merged too, but that should +be coordinate with you. + +```sh +# Run the tests. +bash ./lighthouse-core/scripts/release/test.sh +# Change into the pristine folder. +cd ../lighthouse-pristine +``` + +Confirm DevTools integration will work: +```sh +# You should have Chromium already checked out at ~/chromium/src +# See: https://www.chromium.org/developers/how-tos/get-the-code + +# Roll to Chromium folder. +yarn devtools + +# Checkout latest Chromium code. +cd ~/chromium/src +git pull +git new-branch lh-roll-x.x.x +gclient sync +autoninja -C out/Release chrome blink_tests + +# Run tests and rebase. +yarn --cwd ~/chromium/src/third_party/blink/renderer/devtools test 'http/tests/devtools/audits/*.js' --reset-results +# Verify the changes are expected. +git diff + +# Verify that the Audits panel still works. Consider the new features that have been added. +# If anything is wrong, stop releasing, investigate, and prioritize landing the PR. + +# For bonus points, add some tests covering new features. Either a new test, or an extra +# assertion in an existing test. +``` + +Confirm Lightrider integration will work: +```sh +# Run the internal `update_lighthouse_assets.sh` script. + +# Test things out locally, if happy, deploy to canary and see how the graphs react. + +# Do the stuff in "Test LR changes in Canary". +# https://g3doc.corp.google.com/chrome/headless/lightrider/README.md?cl=head#test-lr-changes-in-canary + +# Verify that the Audits panel still works. Consider the new features that have been added. +# If anything is wrong, stop releasing, investigate, and prioritize landing the PR. + +# For bonus points, add some tests covering new features. Either a new test, or an extra +# assertion in an existing test. +``` + +Now that the integrations are confirmed to work, go back to `lighthouse` folder. + ```sh -# Run the tests +# Run the tests again. +# Note: you can skip this if you didn't need to land any changes. bash ./lighthouse-core/scripts/release/test.sh + # Prepare the commit, replace x.x.x with the desired version bash ./lighthouse-core/scripts/release/prepare-commit.sh x.x.x # Open the PR and await merge... -echo "It's been merged! 🎉" +echo "It's been merged! 🎉" -# Run the tests again :) +# Get that new changelog commit + one last test. bash ./lighthouse-core/scripts/release/test.sh # Package everything for publishing bash ./lighthouse-core/scripts/release/prepare-package.sh @@ -47,13 +130,17 @@ bash ./lighthouse-core/scripts/release/prepare-package.sh # Make sure you're in the Lighthouse pristine repo we just tested. cd ../lighthouse-pristine -# Publish to NPM +# Sanity check: last chance to abort. +git status +git log + +# Publish to npm. npm publish -# Publish viewer +# Publish viewer. yarn deploy-viewer -# Upload the extension +# Publish the extension. open https://chrome.google.com/webstore/developer/edit/blipmdconlkpinefehnmjammfjpmpbjk cd dist/extension-package/ echo "Upload the package zip to CWS dev dashboard..." From 9f0d4a380646cfe575adb5467200fe4d1f123470 Mon Sep 17 00:00:00 2001 From: Connor Clark Date: Wed, 18 Sep 2019 18:00:11 -0700 Subject: [PATCH 02/11] update --- docs/releasing.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/releasing.md b/docs/releasing.md index ab92db200a6a..24a097c36423 100644 --- a/docs/releasing.md +++ b/docs/releasing.md @@ -8,7 +8,6 @@ We aim to release every 3 weeks. Our schedule is set as follows: One day before Example, the next fews releases (as of this writing) are: -* Sep 24 2019 * _Oct 15 2019_ * Nov 5 2019 * _Nov 26 2019_ @@ -18,7 +17,7 @@ Example, the next fews releases (as of this writing) are: * _Feb 18 2020_ * Mar 10 2020 -Underlined dates are the day before the expected Chromium branch point. +Italicized dates are the day before the expected Chromium branch point. The planned ship dates are added to the internal Lighthouse calendar. From 8b42b6b078d7d0789be13a1f2a941efde8d177a3 Mon Sep 17 00:00:00 2001 From: Connor Clark Date: Wed, 18 Sep 2019 18:11:00 -0700 Subject: [PATCH 03/11] fin --- docs/releasing.md | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/docs/releasing.md b/docs/releasing.md index 24a097c36423..9d1a01baec14 100644 --- a/docs/releasing.md +++ b/docs/releasing.md @@ -21,7 +21,9 @@ Italicized dates are the day before the expected Chromium branch point. The planned ship dates are added to the internal Lighthouse calendar. -If a release is necessary outside these scheduled dates, we may choose to skip the next scheduled release. +If a release is necessary outside these scheduled dates, we may choose to skip the next scheduled release. + +In general, the above release dates are when new versions will be available in npm. About a week later, it will be reflected in LR / PSI. Some 10 weeks later, it will be available in Chrome. ### Release manager @@ -90,6 +92,9 @@ git diff # For bonus points, add some tests covering new features. Either a new test, or an extra # assertion in an existing test. + +git cl upload --bypass-hooks +# Go to Gerrit, run CQ dry run, ensure the tests all pass. ``` Confirm Lightrider integration will work: @@ -152,4 +157,17 @@ echo "Upload the package zip to CWS dev dashboard..." # * Tell the world!!! * echo "Complete the _Release publicity_ tasks documented above" + +# Roll the tagged commit to Chromium and update the CL you made. Do not land, see next section. +# Roll the tagged commit to LR and land the CL. ``` + +### Chromium CL + +If this is a branching week, wait until _after_ the branch point email, then land the CL. + +Otherwise, you can land it immediately. + +### The following Monday + +Evaluate LR staging, if all looks good, promote to production! From f1bae92b2c084ba3df8bcdc86d8aa2e9e97bd279 Mon Sep 17 00:00:00 2001 From: Connor Clark Date: Wed, 18 Sep 2019 18:15:01 -0700 Subject: [PATCH 04/11] silly --- docs/releasing.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/releasing.md b/docs/releasing.md index 9d1a01baec14..f1a89ae9a367 100644 --- a/docs/releasing.md +++ b/docs/releasing.md @@ -101,6 +101,8 @@ Confirm Lightrider integration will work: ```sh # Run the internal `update_lighthouse_assets.sh` script. +# Update that silly string that sets the version number for metrics. + # Test things out locally, if happy, deploy to canary and see how the graphs react. # Do the stuff in "Test LR changes in Canary". From 32a808b32f20a1f1d44d8bd4e54262f6fdf30e2a Mon Sep 17 00:00:00 2001 From: Connor Clark Date: Thu, 19 Sep 2019 11:21:52 -0700 Subject: [PATCH 05/11] Update docs/releasing.md Co-Authored-By: Shane Exterkamp --- docs/releasing.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/releasing.md b/docs/releasing.md index f1a89ae9a367..336309f9838a 100644 --- a/docs/releasing.md +++ b/docs/releasing.md @@ -108,7 +108,8 @@ Confirm Lightrider integration will work: # Do the stuff in "Test LR changes in Canary". # https://g3doc.corp.google.com/chrome/headless/lightrider/README.md?cl=head#test-lr-changes-in-canary -# Verify that the Audits panel still works. Consider the new features that have been added. +# Verify that Lightrider works properly, and is generating reports fully. Consider the new features that have been added. +# Note: if the changes include proto changes make sure that the API has those new fields. # If anything is wrong, stop releasing, investigate, and prioritize landing the PR. # For bonus points, add some tests covering new features. Either a new test, or an extra From 1dbab632704635c1c78c1fda9d463b7720454768 Mon Sep 17 00:00:00 2001 From: Connor Clark Date: Thu, 19 Sep 2019 11:33:46 -0700 Subject: [PATCH 06/11] weaker language --- docs/releasing.md | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/docs/releasing.md b/docs/releasing.md index f1a89ae9a367..7d1ff6e175fd 100644 --- a/docs/releasing.md +++ b/docs/releasing.md @@ -6,16 +6,13 @@ We aim to release every 3 weeks. Our schedule is set as follows: One day before the [expected Chromium branch point](https://www.chromium.org/developers/calendar) (which is every six weeks) and again exactly 3 weeks after that day. -Example, the next fews releases (as of this writing) are: +For example, following this schedule, we will attempt a release on these dates: * _Oct 15 2019_ * Nov 5 2019 * _Nov 26 2019_ -* Dec 17 2020 -* _Jan 7 2020_ -* Jan 28 2020 -* _Feb 18 2020_ -* Mar 10 2020 +* Dec 17 2019 +* ... Italicized dates are the day before the expected Chromium branch point. From e11b9c92b9daadc39f0ba74facc4b2888b9c882b Mon Sep 17 00:00:00 2001 From: Connor Clark Date: Mon, 23 Sep 2019 10:12:24 -0700 Subject: [PATCH 07/11] docs --- docs/releasing.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/releasing.md b/docs/releasing.md index 26ef6b857a5c..f7618ad23ba2 100644 --- a/docs/releasing.md +++ b/docs/releasing.md @@ -100,7 +100,7 @@ Confirm Lightrider integration will work: # Update that silly string that sets the version number for metrics. -# Test things out locally, if happy, deploy to canary and see how the graphs react. +# Test things out locally, if happy, deploy to canary and see how the graphs react. 20 minutes should be enough time. # Do the stuff in "Test LR changes in Canary". # https://g3doc.corp.google.com/chrome/headless/lightrider/README.md?cl=head#test-lr-changes-in-canary @@ -126,7 +126,7 @@ bash ./lighthouse-core/scripts/release/prepare-commit.sh x.x.x # Open the PR and await merge... echo "It's been merged! 🎉" -# Get that new changelog commit + one last test. +# One last test (this script uses origin/master, so we also get the commit with the new changelog - that commit should be HEAD). bash ./lighthouse-core/scripts/release/test.sh # Package everything for publishing bash ./lighthouse-core/scripts/release/prepare-package.sh From c54e33d98f33fa97207bffdbda377cf0accf36bf Mon Sep 17 00:00:00 2001 From: Connor Clark Date: Mon, 23 Sep 2019 10:18:31 -0700 Subject: [PATCH 08/11] lr --- docs/releasing.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/releasing.md b/docs/releasing.md index f7618ad23ba2..cdc9fff04914 100644 --- a/docs/releasing.md +++ b/docs/releasing.md @@ -96,9 +96,7 @@ git cl upload --bypass-hooks Confirm Lightrider integration will work: ```sh -# Run the internal `update_lighthouse_assets.sh` script. - -# Update that silly string that sets the version number for metrics. +# See the internal README for updating Lighthouse. # Test things out locally, if happy, deploy to canary and see how the graphs react. 20 minutes should be enough time. From dcb793503b37aa148b5ea1f0e15ea581442cfc08 Mon Sep 17 00:00:00 2001 From: Connor Clark Date: Mon, 23 Sep 2019 10:20:37 -0700 Subject: [PATCH 09/11] rm --- docs/releasing.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/releasing.md b/docs/releasing.md index cdc9fff04914..fe7d60c78f53 100644 --- a/docs/releasing.md +++ b/docs/releasing.md @@ -53,9 +53,7 @@ Note: You'll wanna be on a Linux machine, since the Lightrider step will require ### On the scheduled release date Before starting, you should announce to the LH eng channel that you are releasing, -and that no new PRs should be merged until you are done. If you have to go through all these steps -again to assert that release-blocking fixes work, other PRs may be merged too, but that should -be coordinate with you. +and that no new PRs should be merged until you are done. ```sh # Run the tests. From 5ffd3a656c157f3b4f3b4b1215b6131463b973a7 Mon Sep 17 00:00:00 2001 From: Connor Clark Date: Mon, 23 Sep 2019 10:21:07 -0700 Subject: [PATCH 10/11] docs --- docs/releasing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/releasing.md b/docs/releasing.md index fe7d60c78f53..3032c5c203cd 100644 --- a/docs/releasing.md +++ b/docs/releasing.md @@ -20,7 +20,7 @@ The planned ship dates are added to the internal Lighthouse calendar. If a release is necessary outside these scheduled dates, we may choose to skip the next scheduled release. -In general, the above release dates are when new versions will be available in npm. About a week later, it will be reflected in LR / PSI. Some 10 weeks later, it will be available in Chrome. +In general, the above release dates are when new versions will be available in npm. About a week later, it will be reflected in LR / PSI. Some 10 weeks later, it will be available in Stable Chrome. ### Release manager From 81310d758c9e4eb82aaacbfe54af607193b6b834 Mon Sep 17 00:00:00 2001 From: Connor Clark Date: Fri, 27 Sep 2019 16:37:52 -0700 Subject: [PATCH 11/11] pr --- docs/releasing.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/releasing.md b/docs/releasing.md index 3032c5c203cd..51f42e2c9dc8 100644 --- a/docs/releasing.md +++ b/docs/releasing.md @@ -1,5 +1,7 @@ ### Release guide for maintainers +This doc is only relevant to core member. + ## Release Policy ### Cadence @@ -99,7 +101,7 @@ Confirm Lightrider integration will work: # Test things out locally, if happy, deploy to canary and see how the graphs react. 20 minutes should be enough time. # Do the stuff in "Test LR changes in Canary". -# https://g3doc.corp.google.com/chrome/headless/lightrider/README.md?cl=head#test-lr-changes-in-canary +# go/lightrider-doc#test-lr-changes-in-canary # Verify that Lightrider works properly, and is generating reports fully. Consider the new features that have been added. # Note: if the changes include proto changes make sure that the API has those new fields.