[No QA] Build Cherry-pick flow for Expensify.cash

[roryabraham]

Note: This PR creates a CP-staging workflow only. Not CP-production.

Details

Creates the ability to cherry-pick a workflow to staging by merging a pull request with the CP Staging label.

Fixed Issues

Fixes https://github.com/Expensify/Expensify/issues/155229

Testing plan

1. Ensure that the StagingDeployCash is not locked

2. Create a pull request with the CP Staging label. OSBotify should comment on the pull request with a comment that looks like:

   

3. Merge the pull request with the CP Staging label. It should follow the normal process and be deployed to staging.

4. Lock the StagingDeployCash. The normal chain-of-events should happen, culminating with a staging deploy. The cherryPick.yml workflow should not be executed.

5. Merge a pull request with an easily identifiable change (no CP Staging label). Call this PR A. As per usual, it should not be deployed or added to the StagingDeployCash. A comment should appear on the PR stating that it will be deployed later.

6. @Jag96 - Merge a pull request with an easily identifiable change with the CP Staging label on the PR. Call this PR B. A new version should be created and merged to main, and the preDeploy workflow should synchronously execute the cherryPick.yml workflow.

7. A staging deploy should occur, and a comment should appear on PR B stating that it was cherry picked (not just deployed) to staging in the correct version.

8. Once the staging deploy completes, verify that PR A is not present on staging, but PR B is. This is where the easily-identifiable changes will come in handy.

9. Verify that PR B was added to the StagingDeployCash, but PR A was not.

10. @Jag96 - Use the GitHub UI to attempt to manually CP PR A to staging. The workflow should not complete. Verify that no staging deploy happens.

11. @AndrewGable or @roryabraham - Use the GitHub UI to attempt to manually CP PR A to staging. It should work and a staging deploy should occur.

12. Once the staging deploy completes, verify that PR A is present on staging.

13. Verify that PR A was added to the StagingDeployCash.

14. Unlock the StagingDeployCash and revert any temporary changes.

[AndrewGable]

I know this is a late to the game comment, but did we explore adding the ability to add the label after the PR is merged? Or will that be able to be added in a future iteration?

[roryabraham]

did we explore adding the ability to add the label after the PR is merged

Yeah, so this was designed with a specific vision in mind. Basically in many places we currently have if: github.actor == 'OSBotify', we'll instead use an action to check that the actor is a member of a Expensify.cash Deployers GH group. That will allow certain privileged individuals to manually run/restart workflows that currently can only be run by OSBotify.

With that in mind, the cherryPick.yml workflow runs on the workflow_dispatch event, which means it can only be started "manually". So what we'll be able to do is manually trigger the cherryPick.yml workflow using the GH UI, and all you'll have to do is provide the number of the pull request you want to CP. It would look something like this:

[AndrewGable]

Wow I had no idea that UI existed, looks great!

[Jag96]

Looking good, just 2 comments

[roryabraham]

If a PR is created with the CP Staging, let's add a comment like:

⚠️ ⚠️ Heads up! This pull request has the CP Staging label. ⚠️ ⚠️
Merging it will cause it to be immediately deployed to staging, even if the open StagingDeployCash deploy checklist is locked.

[roryabraham]

Okay, this is updated and ready for another review. Added some botify comments and opened up the deployBlocker.yml workflow for all members of the Expensify.cash Deployers GH Team

[Jag96]

LGTM 👍 Let me know if you need help testing this!

[AndrewGable]

Looks great, feel free to test without me (or if I am around I am happy to help testing!)

[jasperhuangg]

Looks good! I don't have the most knowledge about this stuff so I followed as best as I could. Feel free to merge once the test that @AndrewGable mentioned is added!

[roryabraham]

Marking this as No QA for Applause's sake, but will begin testing this now.

[roryabraham]

• StagingDeployCash is not locked
• PR with CP label gets warning comment: [No QA] Move GitHub README from .github/workflows/ to .github/ #3144
• Merging PR w/ CP label results in normal staging deploy: https://github.com/Expensify/Expensify.cash/runs/2670956086?check_suite_focus=true
• Merged PR gets added to deploy checklist: Deploy Checklist: Expensify.cash 2021-05-26 #3143
• Locking StagingDeployCash results in PATCH version bump and staging deploy: https://github.com/Expensify/Expensify.cash/actions/runs/876957376
• Regular PR (PR A) which is merged when StagingDeployCash is locked is deferred and not deployed to staging: Make easily noticeable temporary change #3149, https://github.com/Expensify/Expensify.cash/actions/runs/876993264
• PR w/ CP Staging label (PR B) is CP'd: [CP Testing] Change blue to a vibrant magenta #3150, https://github.com/Expensify/Expensify.cash/runs/2671108199?check_suite_focus=true ❌

[roryabraham]

Fixing the last failure with this PR

[roryabraham]

Adjusting the test plan a bit: we are going to:

1. skip steps 6-9 for now
2. Complete steps 10-11, except use PR B
3. Complete step 7-9
4. Complete steps 6-9 again with the revert PR for PR B

[roryabraham]

Okay, so @Jag96 tried to manually CP this PR, but we ran into a permissions error where the OSBotify token needs higher permissions.

[roryabraham]

Okay, this testing was unfortunately a failure, and we need to step back and get some input from ring 0 folks to continue. For now, we'll:

1. Revert [CP Testing] Change blue to a vibrant magenta #3150
2. Unlock the StagingDeployCash
3. Revert Make easily noticeable temporary change #3149
4. Continue discussion over here

[roryabraham]

Continuing QA on this issue with an updated plan: since we already verified steps 1-3, we're going to restart at step 4:

4. Lock the StagingDeployCash. The normal chain-of-events should happen, culminating with a staging deploy. The cherryPick.yml workflow should not be executed.
5. Merge this PR: Temporarily replace blue with magenta #3209. As per usual, it should not be deployed or added to the StagingDeployCash. A comment should appear on the PR stating that it will be deployed later.
6. Someone other than @AndrewGable or I: merge this PR: Temporarily replace green with cyan #3208. A new BUILD version should be created and merged to main, and the preDeploy workflow should synchronously execute the cherryPick.yml workflow.
7. A staging deploy should occur, and a comment should appear on Temporarily replace green with cyan #3208 stating that it was cherry picked (not just deployed) to staging in the correct version.
8. Once the staging deploy completes, verify that Temporarily replace blue with magenta #3209 is not present on staging, but Temporarily replace green with cyan #3208 is. This is where the easily-identifiable changes will come in handy.
9. Verify that Temporarily replace green with cyan #3208 was added to the StagingDeployCash, but Temporarily replace blue with magenta #3209 was not.
10. Someone other than @AndrewGable or I: Use the GitHub UI to attempt to manually CP Temporarily replace blue with magenta #3209 to staging. The workflow should not complete. Verify that no staging deploy happens.
11. @AndrewGable or I: Use the GitHub UI to attempt to manually CP Temporarily replace blue with magenta #3209 to staging. It should work and a staging deploy should occur.
12. Once the staging deploy completes, verify that Temporarily replace blue with magenta #3209 is present on staging.
13. Create revert-PRs for both test PRs, give both the CP Staging label, and merge both.
14. Once the later staging deploy completes, verify that there are no incorrect colors on staging.
15. Close the StagingDeployCash to run a prod deploy. Verify that the latest staging version is deployed to prod. We'll have locked the current StagingDeployCash and CP'd 4 PRs, so I expect that version to be 1.0.57-4.

[roryabraham]

4. https://github.com/Expensify/Expensify.cash/runs/2696159849?check_suite_focus=true ✅
5. https://github.com/Expensify/Expensify.cash/runs/2696167506?check_suite_focus=true ✅
   • Note: we discovered a race condition while testing this, so Temporarily replace blue with magenta #3209 was deployed to staging. Should be fixed in [No QA] Use full Git history to populate StagingDeployCash #3173, but for now we'll need to adjust our testing plan a bit. NBD, nothing seriously broken yet.
6. https://github.com/Expensify/Expensify.cash/runs/2696535209?check_suite_focus=true, https://github.com/Expensify/Expensify.cash/runs/2696547589?check_suite_focus=true ❌
   • CP actor validation didn't seem to work, and the logs are not very helpful.
   • Note: we probably don't want to do the "deploy deferred" comment when a PR has the CP Staging label