GitHub Actions (#854)

* Add test workflow

* Use actions/checkout v3

* Don't use CI image

* Add step install npm dependencies

* Add setup action

* Set DEVELOP and RELEASE env variables

* Add required shell property

* Update action.yml

* Update action.yml

* Adding test matrix

* cleanup

* Split setup windows and ubuntu

* Update test.yml

* Update test.yml

* Add Lighthouse and smoketest scripts

* Not using actions/cache for now

* Add generated matrix

* Add missing runs-on prop to generated matrix

* Add Setup Node step to generated matrix

* Add Setup machine to generated matrix

* Add cron schedule

* Add timeout-minutes 5

* Move timeout to generate project step

* Add Run tests on generated projects

* Use dynamic generated-project folder

* Run tests on test-project and retail-react-app-demo

* Add Push Bundle step

* Skip flaky test

* Disable fail-fast strategy

* Use env variables

* Re-arrange env

* Add step before push bundle

* cleanup

* cleanup

* Use temp test-env-3

* testing slack notifs

* testing

* add publish to npm step

* fix indent

* python-dev does not exist anymore

* use python2

* increase max packages

* test slack notifs

* add snyk cli and datadog step

* update mrt user credentials

* testing slack with pwa kit channel

* syntax

* fix conditionals

* test push bundle

* add push bundle step for generated

* syntax

* fix syntax error

* update slack payload

* run steps in container

* testing

* refactor

* syntax

* sudo container error

* testing

* update

* add pip

* use different docker

* no container

* container

* testing

* add user to container

* fix

* syntax add shell

* syntax errors

* remove container, use act

* syntax errors

* add snyk audit and other syntax stuff

* extract steps to own actions

* add inputs for actions

* add shell for steps in actions with run

* project cannot be generated in action file

* updated snyk token, uncommenting code

* Fix typo.

* Add missing appkey property.

* Use snake_case names for legibility.

* Restore missing clean check

* Fix skipped conversion to snake_case.

* Trim trailing whitespace.

* Extract conditionals to vars and clean up vars.

* Change env IS_TESTABLE_TEMPLATE to more clear IS_TEMPLATE_FROM_RETAIL_REACT_APP

* Fix YAML breaking conditional.

* Try explicitly checking value.

* Try explicitly checking true/false string values.

* Try string comparisons.

* Fix bad YAML.

* Replace " with '

* Get ready for the prime time!

* Fail fast!

* Update TODOs.

* Clean up npm version management.

* Add TODO to merge workflows.

* Update step names.

* End files with newline.

* Run on pull_request to support forked repos.

* Only run on push for develop/release.

We can assume all other branches will eventually have a PR.

* Only push to MRT when actually desired.

* Get that JavaScript nonsense outta here!

* Check DEVELOP in step conditional, not in action execution.

* Add some TODOs.

* Too many newlines!

Co-authored-by: yunakim714 <[email protected]>
Co-authored-by: yunakim714 <[email protected]>
Co-authored-by: Will Harney <[email protected]>
Co-authored-by: Will Harney <[email protected]>

.github/actions/check_clean/action.yml

@@ -0,0 +1,12 @@
+name: check_clean
+runs:
+  using: composite
+  steps:
+  - name: Check Repository Clean
+    run: |-
+      # Print status for debugging on CircleCI.
+      git status
+      # Fail the build if any step leaves uncommitted changes in the repo
+      # that would prevent Lerna from publishing (Lerna gets this right).
+      git diff --exit-code
+    shell: bash

.github/actions/count_deps/action.yml

@@ -0,0 +1,18 @@
+name: count_deps
+runs:
+  using: composite
+  steps:
+  - name: Count Generated Project Dependencies
+    # TODO: Can TOTAL_PACKAGES be exported in a cleaner way?
+    run: |-
+      MAX_PACKAGES="2260"
+      total=$(./scripts/count-dependencies.js generated-${{ matrix.template }})
+      echo "TOTAL_PACKAGES=${total}" >> $GITHUB_ENV
+
+      if [ "$total" -gt "$MAX_PACKAGES" ]; then
+          echo "Error: Found $TOTAL_PACKAGES installed packages (max $MAX_PACKAGES).";
+          exit 1;
+      else
+          echo "Found $TOTAL_PACKAGES installed packages (max $MAX_PACKAGES).";
+      fi
+    shell: bash

.github/actions/create_mrt/action.yml

@@ -0,0 +1,14 @@
+name: create_mrt
+inputs:
+  mobify_user:
+    description: "Mobify user email"
+  mobify_api_key:
+    description: "Mobify user API key"
+runs:
+  using: composite
+  steps:
+    - name: Create MRT credentials file
+      run: |-
+        # Add credentials file at ~/.mobify so we can upload to Mobify Cloud
+        npm run save-credentials --prefix packages/template-retail-react-app -- --user "${{inputs.mobify_user}}" --key "${{inputs.mobify_api_key}}"
+      shell: bash

.github/actions/datadog/action.yml

@@ -0,0 +1,16 @@
+name: datadog
+inputs:
+  datadog_api_key:
+    description: "Datadog API key"
+  TOTAL_PACKAGES:
+    description: "Total # of packages"
+runs:
+  using: composite
+  steps:
+    - name: Send metrics to Datadog
+      run : |
+        # Add a dogrc so we can submit metrics to datadog
+        printf "[Connection]\napikey = ${{inputs.datadog_api_key}}\nappkey =\n" > ~/.dogrc
+
+        dog metric post mobify_platform_sdks.generated_project_total_packages ${{ inputs.TOTAL_PACKAGES }}
+      shell: bash

.github/actions/lighthouse_ci/action.yml

@@ -0,0 +1,7 @@
+name: lighthouse_ci
+runs:
+  using: composite
+  steps:
+  - name: Run Lighthouse CI on the PWA
+    run: npm run test:lighthouse --prefix packages/template-retail-react-app
+    shell: bash

.github/actions/publish_to_npm/action.yml

@@ -0,0 +1,21 @@
+name: publish_to_npm
+inputs:
+  NODE_AUTH_TOKEN:
+    description: "Node auth token"
+runs:
+  using: composite
+  steps:
+  # TODO: Figure out a way to specify whether to publish to "latest" or "next" tag
+  - name: Publish to NPM
+    run: |-
+      # Add NPM token to allow publishing
+      echo "//registry.npmjs.org/:_authToken=${{ inputs.NODE_AUTH_TOKEN }}" > ~/.npmrc
+
+      # Publish all changed packages. The "from-package" arg means "look
+      # at the version numbers in each package.json file and if that doesn't
+      # exist on NPM, publish"
+      npm run lerna -- publish from-package --yes --no-verify-access
+
+      # Cleanup
+      rm ~/.npmrc
+    shell: bash

.github/actions/push_to_mrt/action.yml

@@ -0,0 +1,18 @@
+name: push_to_mrt
+inputs:
+  CWD:
+    description: Project directory
+  TARGET:
+    description: MRT target
+runs:
+  using: composite
+  steps:
+    - name: Push Bundle to MRT
+      run: |-
+        cd ${{ inputs.CWD }}
+        project="scaffold-pwa"
+        build="build ${{ github.run_id }} on ${{ github.ref }} (${{ github.sha }})"
+        if [[ ${{ inputs.TARGET }} ]]; then
+          npm run push -- -s $project --message "$build" --target ${{ inputs.TARGET }}
+        fi
+      shell: bash

.github/actions/setup_ubuntu/action.yml

@@ -0,0 +1,35 @@
+name: setup_ubuntu
+inputs:
+  cwd:
+    required: false
+    default: "${PWD}"
+description: "Setup Ubuntu Machine"
+runs:
+  using: composite
+  steps:
+    - name: Install Dependencies
+      run: |-
+        # Install system dependencies
+        sudo apt-get update -yq
+        sudo apt-get install python2 python3-pip time -yq
+        sudo pip install -U pip setuptools
+        sudo pip install awscli==1.18.85 datadog==0.40.1
+
+        # Install node dependencies
+        node ./scripts/gtime.js monorepo_install npm ci
+
+        # Build the PWA
+        npm run lerna -- run analyze-build --scope "retail-react-app"
+
+        # Report bundle sizes
+        node ./scripts/report-bundle-size.js
+
+        # Check that packages are all using the same versions of compilers, etc.
+        node ./scripts/check-dependencies.js
+
+        # Install Snyk CLI
+        sudo npm install -g snyk
+
+        # Install Lighthouse CI CLI
+        sudo npm install -g @lhci/cli
+      shell: bash

.github/actions/setup_windows/action.yml

@@ -0,0 +1,14 @@
+name: setup_windows
+inputs:
+  cwd:
+    required: false
+    default: "${PWD}"
+description: "Setup Windows Machine"
+runs:
+  using: composite
+  steps:
+    - name: Install Dependencies
+      run: |-
+        # Install node dependencies
+        npm ci
+      shell: bash

.github/actions/smoke_tests/action.yml

@@ -0,0 +1,14 @@
+name: smoke_tests
+inputs:
+  dir:
+    required: false
+    # The path to a project to test
+    default: "./packages/template-retail-react-app"
+runs:
+  using: composite
+  steps:
+  - name: Smoke test scripts
+    run: |-
+      # Basic smoke-tests for uncommonly run scripts in a project
+      node ./scripts/smoke-test-npm-scripts.js --dir ${{ inputs.dir }}
+    shell: bash

.github/actions/snyk/action.yml

@@ -0,0 +1,15 @@
+name: snyk
+inputs:
+  snyk_token:
+    description: "Snyk token"
+  DEVELOP:
+    description: "Is this the 'develop' branch?"
+runs:
+  using: composite
+  steps:
+  - name: Audit Generated Project
+    run: |-
+        # Run snyk auth - authenticate snyk using environment variables to add the token
+        snyk auth ${{ inputs.snyk_token }}
+        snyk monitor --ignore-policy --remote-repo-url='https://github.com/SalesforceCommerceCloud/pwa-kit.git' --project-name='generated-scaffold-pwa'
+    shell: bash

.github/actions/unit_tests/action.yml

@@ -0,0 +1,27 @@
+name: unit_tests
+inputs:
+  cwd:
+    required: false
+    default: "${PWD}"
+description: "Run tests action description"
+runs:
+  using: composite
+  steps:
+  - name: Run tests step
+    # TODO: The pilefile policy is a legacy of CircleCI. Is it still needed?
+    run: |-
+      # Explicitly set pipefile policy. This is the default for non-windows, but seems
+      # that is needs to be set on windows to fail immediately.
+      set -eo pipefail
+
+      cd ${{ inputs.cwd }}
+
+      # Note: Each of these test commands need to be exposed on the monorepo
+      # root and *also* on the PWA package. This section is run on both.
+
+      # Ensure bundlesize is in check
+      npm run test:max-file-size
+
+      # Always run fast unit tests
+      npm run test
+    shell: bash