diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index c0b3e9e16f75..79cd564f63ba 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -294,6 +294,9 @@ Here is a list of the npm packages in this repository: | Folder Name | Package Name | Purpose | | :----------------------------------------------------- | :--------------------------------- | :--------------------------------------------------------------------------- | + | [eslint-plugin-dev](./npm/eslint-plugin-dev) | `@cypress/eslint-plugin-dev` | Eslint plugin for internal development. | + | [react](./npm/react) | `@cypress/react` | Cypress component testing for React. | + | [vue](./npm/vue) | `@cypress/vue` | Cypress component testing for Vue. | | [webpack-preprocessor](./npm/webpack-preprocessor) | `@cypress/webpack-preprocessor` | Cypress preprocessor for bundling JavaScript via webpack. | We try to tag all issues with a `pkg/` or `npm/` tag describing the appropriate package the work is required in. For public packages, we use their qualified package name: For example, issues relating to the webpack preprocessor are tagged under [`npm: @cypress/webpack-preprocessor`](https://github.com/cypress-io/cypress/labels/npm%3A%20%40cypress%2Fwebpack-preprocessor) label and issues related to the `driver` package are tagged with the [`pkg/driver`](https://github.com/cypress-io/cypress/labels/pkg%2Fdriver) label. @@ -356,6 +359,8 @@ $ yarn workspace @packages/server add my-new-dep1 $ yarn workspace @packages/server add --dev my-new-dep1 ``` +Alternatively, you can directly add the dependency to the corresponding `package.json`. + #### Tasks > Scripts are intended to be **run** from the **root of the repo**. **Do not install dependencies or run scripts from within a sub-directory.** @@ -439,7 +444,7 @@ DEBUG=cypress:launcher ### Coding Style We use [eslint](https://eslint.org/) to lint all JavaScript code and follow rules specified in -[@cypress/eslint-plugin-dev](https://github.com/cypress-io/eslint-plugin-cypress) plugin. +[@cypress/eslint-plugin-dev](./npm/eslint-plugin-cypress) plugin. When you edit files, you can quickly fix all changed files before you commit using @@ -464,14 +469,18 @@ This is to ensure that links do not go dead in older versions of Cypress when th ### Tests -For most packages there are typically unit and some integration tests. +For most packages there are typically unit and integration tests. Our true e2e tests are in [`packages/server`](packages/server), which test the full stack all together. +Additionally, we test the code by running it against various other example projects in CI. See CI badges and links at the top of this document. + Please refer to each packages' `README.md` which documents how to run tests. It is not feasible to try to run all of the tests together. We run our entire test fleet across over a dozen containers in CI. If you're curious how we manage all of these tests in CI check out our [`circle.yml`](circle.yml) file found in the root `cypress` directory. +Each of the independent packages (in the [`/npm`](./npm) folder) have a `ciJobs` field in their `package.json`. This field corresponds to the CI jobs for that package and is used when determining what tests must pass before the package can be released. + #### Docker Sometimes tests pass locally, but fail in CI. Our CI environment is dockerized. In order to run the image used in CI locally: @@ -559,6 +568,52 @@ All updates to `master` are automatically merged into `develop`, so `develop` al +### Independent Packages CI Workflow + +Independent packages are automatically released when code is merged into `master`. In order to make these automatic releases work smoothly, independent packages have a couple of special configuration options in their `package.json`. + +##### `ciJobs` + +List of Circle CI jobs that directly test the current package. These tests must pass before the package can be released. + +In addition, these tests will run when a PR is created that changes this package. All tests will run on `develop` and `master`, regardless of what packages were changed. + +Note: CI jobs should be unique to a package. Any jobs that are not listed within a `ciJobs` field are considered to be part of the binary and will only run when the binary is changed. + +This option takes an array of CI job names. + +Example +```json +{ + "ciJobs": [ + "npm-react", + "npm-react-axe", + "npm-react-next" + ] +} +``` + +##### `ciDependents` + +List of local independent (npm) packages that are dependent on the current package. The tests specified in these packages' `ciJobs` must pass before the current package will be released. + +When the current package is changed in a PR, it will consider these packages to be changed as well and run CI accordingly. + +This option takes an array of package names. + +Example +```json +{ + "ciDependents": [ + "@cypress/react", + "@cypress/vue", + "@cypress/webpack-preprocessor" + ] +} +``` + +You can read more about our CI design decisions in [#8730](https://github.com/cypress-io/cypress/pull/8730#issue-496593325) + ### Pull Requests - When opening a PR for a specific issue already open, please name the branch you are working on using the convention `issue-[issue number]`. For example, if your PR fixes Issue #803, name your branch `issue-803`. If the PR is a larger issue, you can add more context like `issue-803-new-scrollable-area` If there is not an associated open issue, **create an issue using our [Issue Template](./.github/ISSUE_TEMPLATE.md)**. @@ -568,12 +623,6 @@ All updates to `master` are automatically merged into `develop`, so `develop` al - Please check the "Allow edits from maintainers" checkbox when submitting your PR. This will make it easier for the maintainers to make minor adjustments, to help with tests or any other changes we may need. ![Allow edits from maintainers checkbox](https://user-images.githubusercontent.com/1271181/31393427-b3105d44-ada9-11e7-80f2-0dac51e3919e.png) -### Testing - -This repository is exhaustively tested by [CircleCI](https://circleci.com/gh/cypress-io/cypress). Additionally we test the code by running it against various other example projects. See CI badges and links at the top of this document. - -To run local tests, consult the `README.md` of each package. - ### Dependencies We use [RenovateBot](https://renovatebot.com/) to automatically upgrade our dependencies. The bot uses the settings in [renovate.json](renovate.json) to maintain our [Update Dependencies](https://github.com/cypress-io/cypress/issues/3777) issue and open PRs. You can manually select a package to open a PR from our [Update Dependencies](https://github.com/cypress-io/cypress/issues/3777) issue. @@ -651,7 +700,9 @@ Below are some guidelines Cypress uses when reviewing dependency updates. ## Deployment -We will try to review and merge pull requests quickly. After merging we will try releasing a new version. If you want to know our build process or build your own Cypress binary, read [DEPLOY.md](./DEPLOY.md) +We will try to review and merge pull requests quickly. If you want to know our build process or build your own Cypress binary, read [DEPLOY.md](./DEPLOY.md). + +Independent packages are deployed immediately upon being merged into master. You can read more [above](#independent-packages-ci-workflow). ## Known problems diff --git a/DEPLOY.md b/DEPLOY.md index 3d3150092922..d4c4d86d6cbc 100644 --- a/DEPLOY.md +++ b/DEPLOY.md @@ -1,6 +1,8 @@ # Deployment -These deployment procedures mainly concern the Cypress binary and `cypress` npm module. Independent `@cypress/` packages that live inside the [`npm`](./npm) directory are automatically published to npm (with [`semantic-release`](https://semantic-release.gitbook.io/semantic-release/)) upon being merged into master. +These deployment procedures mainly concern the Cypress binary and `cypress` npm module. + +Independent `@cypress/` packages that live inside the [`npm`](./npm) directory are automatically published to npm (with [`semantic-release`](https://semantic-release.gitbook.io/semantic-release/)) upon being merged into master. You can read more about this in [CONTRIBUTING.md](./CONTRIBUTING.md#committing-code) Anyone can build the binary and npm package, but you can only deploy the Cypress application and publish the npm module `cypress` if you are a member of the `cypress` npm organization. diff --git a/circle.yml b/circle.yml index 2d49c0c14a6d..8347b98956ff 100644 --- a/circle.yml +++ b/circle.yml @@ -120,6 +120,7 @@ commands: steps: - attach_workspace: at: ~/ + - check-halt - run: environment: CYPRESS_KONFIG_ENV: production @@ -161,6 +162,7 @@ commands: steps: - attach_workspace: at: ~/ + - check-halt - run: command: | cmd=$([[ <> == 'true' ]] && echo 'yarn percy exec --') || true @@ -184,6 +186,7 @@ commands: steps: - attach_workspace: at: ~/ + - check-halt - run: command: yarn workspace @packages/server test ./test/e2e/$(( $CIRCLE_NODE_INDEX ))_*spec* --browser <> - verify-mocha-results @@ -281,6 +284,7 @@ commands: steps: - attach_workspace: at: ~/ + - check-halt # make sure the binary and NPM package files are present - run: ls -l - run: ls -l cypress.zip cypress.tgz @@ -381,6 +385,13 @@ commands: name: "Waiting on Circle CI jobs: <>" command: node ./scripts/wait-on-circle-jobs.js --job-names="<>" + check-halt: + description: Halt CI if the package that this job corresponds to is unchanged + steps: + - run: + name: Check if job should run + command: yarn check-halt || circleci-agent step halt + jobs: ## code checkout and yarn installs build: @@ -459,6 +470,7 @@ jobs: root: ~/ paths: - cypress + - .ssh lint: <<: *defaults @@ -478,6 +490,15 @@ jobs: command: node cli/bin/cypress info --dev - store-npm-logs + list-changed-packages: + <<: *defaults + steps: + - attach_workspace: + at: ~/ + - run: + name: List changed packages + command: node scripts/changed-packages.js + # a special job that keeps polling Circle and when all # individual jobs are finished, it closes the Percy build percy-finalize: @@ -515,6 +536,7 @@ jobs: steps: - attach_workspace: at: ~/ + - check-halt - run: mkdir -p cli/visual-snapshots - run: command: node cli/bin/cypress info --dev | yarn --silent term-to-html | node scripts/sanitize --type cli-info > cli/visual-snapshots/cypress-info.html @@ -539,6 +561,7 @@ jobs: steps: - attach_workspace: at: ~/ + - check-halt # make sure mocha runs - run: yarn test-mocha # test binary build code @@ -584,6 +607,7 @@ jobs: steps: - attach_workspace: at: ~/ + - check-halt - install-required-node - run: name: Mocha tests @@ -603,6 +627,7 @@ jobs: steps: - attach_workspace: at: ~/ + - check-halt - run: yarn test-unit --scope @packages/server - verify-mocha-results: expectedResultCount: 1 @@ -616,6 +641,7 @@ jobs: steps: - attach_workspace: at: ~/ + - check-halt - run: yarn test-integration --scope @packages/server - verify-mocha-results: expectedResultCount: 1 @@ -628,6 +654,7 @@ jobs: steps: - attach_workspace: at: ~/ + - check-halt - run: command: yarn workspace @packages/server test-performance - verify-mocha-results: @@ -708,6 +735,7 @@ jobs: steps: - attach_workspace: at: ~/ + - check-halt - run: command: yarn build-prod working_directory: packages/desktop-gui @@ -733,6 +761,7 @@ jobs: steps: - attach_workspace: at: ~/ + - check-halt - run: command: yarn build-prod working_directory: packages/desktop-gui @@ -767,6 +796,7 @@ jobs: steps: - attach_workspace: at: ~/ + - check-halt - run: # builds JS and CSS, and we need the app CSS # to correctly apply component styles @@ -800,6 +830,7 @@ jobs: steps: - attach_workspace: at: ~/ + - check-halt - run: command: yarn build-for-tests working_directory: packages/reporter @@ -821,6 +852,7 @@ jobs: steps: - attach_workspace: at: ~/ + - check-halt - run: command: yarn build-for-tests working_directory: packages/ui-components @@ -842,6 +874,7 @@ jobs: steps: - attach_workspace: at: ~/ + - check-halt - run: command: node index.js working_directory: packages/launcher @@ -851,6 +884,7 @@ jobs: steps: - attach_workspace: at: ~/ + - check-halt - run: name: Build command: yarn workspace @cypress/webpack-preprocessor build @@ -884,23 +918,12 @@ jobs: working_directory: npm/webpack-preprocessor/examples/react-app - store-npm-logs - npm-webpack-preprocessor-release: - <<: *defaults - steps: - - attach_workspace: - at: ~/ - - run: - name: Build - command: yarn workspace @cypress/webpack-preprocessor build - - run: - name: Release - command: yarn semantic-release @cypress/webpack-preprocessor - npm-vue: <<: *defaults steps: - attach_workspace: at: ~/ + - check-halt - run: name: Build command: yarn workspace @cypress/vue build @@ -909,20 +932,12 @@ jobs: command: yarn workspace @cypress/vue test - store-npm-logs - npm-vue-release: - <<: *defaults - steps: - - attach_workspace: - at: ~/ - - run: - name: Release - command: yarn semantic-release @cypress/vue - npm-react: <<: *defaults steps: - attach_workspace: at: ~/ + - check-halt - run: name: Build command: yarn workspace @cypress/react build @@ -951,6 +966,7 @@ jobs: steps: - attach_workspace: at: ~/ + - check-halt - restore_cache: name: Restore Cache keys: @@ -969,32 +985,24 @@ jobs: command: yarn test working_directory: npm/react/<> - npm-react-release: - <<: *defaults - steps: - - attach_workspace: - at: ~/ - - run: - name: Release - command: yarn semantic-release @cypress/react - npm-eslint-plugin-dev: <<: *defaults steps: - attach_workspace: at: ~/ + - check-halt - run: name: Run tests command: yarn workspace @cypress/eslint-plugin-dev test - npm-eslint-plugin-dev-release: + npm-release: <<: *defaults steps: - attach_workspace: at: ~/ - run: - name: Release - command: yarn semantic-release @cypress/eslint-plugin-dev + name: Release packages + command: yarn npm-release build-binary: <<: *defaults @@ -1029,6 +1037,7 @@ jobs: - attach_workspace: at: ~/ + - check-halt - run: $(yarn bin)/print-arch - install-required-node - run: @@ -1061,6 +1070,7 @@ jobs: steps: - attach_workspace: at: ~/ + - check-halt - run: ls -l - run: name: upload unique binary @@ -1078,6 +1088,7 @@ jobs: test-kitchensink: <<: *defaults steps: + - check-halt - clone-repo-and-checkout-release-branch: repo: cypress-example-kitchensink - install-required-node @@ -1104,6 +1115,7 @@ jobs: "test-kitchensink-against-staging": <<: *defaults steps: + - check-halt - clone-repo-and-checkout-release-branch: repo: cypress-example-kitchensink - run: @@ -1128,6 +1140,7 @@ jobs: "test-against-staging": <<: *defaults steps: + - check-halt - clone-repo-and-checkout-release-branch: repo: cypress-test-tiny - run: @@ -1144,6 +1157,7 @@ jobs: steps: - attach_workspace: at: ~/ + - check-halt - install-required-node - run: name: Check next dev version @@ -1190,6 +1204,7 @@ jobs: steps: - attach_workspace: at: ~/ + - check-halt - run: ls -l # NPM package file should have filename cypress-.tgz - run: @@ -1212,6 +1227,7 @@ jobs: # needs uploaded NPM and test binary - attach_workspace: at: ~/ + - check-halt - run: ls -la # make sure JSON files with uploaded urls are present - run: ls -la binary-url.json npm-package-url.json @@ -1249,6 +1265,7 @@ jobs: # needs uploaded NPM and test binary - attach_workspace: at: ~/ + - check-halt - run: ls -la - post-install-comment @@ -1257,6 +1274,7 @@ jobs: steps: - attach_workspace: at: ~/ + - check-halt # make sure we have cypress.zip received - run: ls -l - run: ls -l cypress.zip cypress.tgz @@ -1292,6 +1310,7 @@ jobs: steps: - attach_workspace: at: ~/ + - check-halt # make sure we have cypress.zip received - run: ls -l - run: ls -l cypress.zip cypress.tgz @@ -1333,6 +1352,7 @@ jobs: steps: - attach_workspace: at: ~/ + - check-halt # make sure we have cypress.zip received - run: ls -l - run: ls -l cypress.zip cypress.tgz @@ -1376,6 +1396,7 @@ jobs: steps: - attach_workspace: at: ~/ + - check-halt # make sure we have cypress.zip received - run: ls -l - run: ls -l cypress.zip cypress.tgz @@ -1419,6 +1440,7 @@ jobs: steps: - attach_workspace: at: ~/ + - check-halt # make sure we have cypress.zip received - run: ls -l - run: ls -l cypress.zip cypress.tgz @@ -1451,6 +1473,7 @@ jobs: steps: - attach_workspace: at: ~/ + - check-halt - run: ls -l # make sure we have the binary and NPM package - run: ls -l cypress.zip cypress.tgz @@ -1586,6 +1609,7 @@ jobs: steps: - attach_workspace: at: ~/ + - check-halt # the user should be "node" - run: whoami - run: pwd @@ -1636,6 +1660,9 @@ linux-workflow: &linux-workflow name: Linux lint requires: - build + - list-changed-packages: + requires: + - build - percy-finalize: context: test-runner:poll-circle-workflow required_env_var: PERCY_TOKEN # skips job if not defined (external PR) @@ -1712,69 +1739,50 @@ linux-workflow: &linux-workflow - npm-webpack-preprocessor: requires: - build - - npm-webpack-preprocessor-release: - filters: - branches: - only: - - master - requires: - - build - - npm-webpack-preprocessor - - npm-vue: requires: - build - - npm-vue-release: - filters: - branches: - only: - - master - requires: - - build - - npm-vue - - npm-react: requires: - build - # Run tests for end-to-end react component tests examples - npm-react-e2e-example: - name: React + Axe component testing + name: npm-react-axe path: examples/a11y requires: - npm-react - npm-react-e2e-example: - name: React + Next component testing + name: npm-react-next path: examples/nextjs requires: - npm-react - npm-react-e2e-example: - name: React + CRA component testing + name: npm-react-cra path: examples/react-scripts requires: - npm-react - npm-react-e2e-example: - name: React + CRA folder component testing + name: npm-react-cra-folder path: examples/react-scripts-folder requires: - npm-react - npm-react-e2e-example: - name: React + Rollup component testing + name: npm-react-rollup path: examples/rollup requires: - npm-react - npm-react-e2e-example: - name: React + Sass + TS component testing + name: npm-react-sass-ts path: examples/sass-and-ts requires: - npm-react - npm-react-e2e-example: - name: React + Snapshots component testing + name: npm-react-snapshots path: examples/snapshots requires: - npm-react - npm-react-e2e-example: - name: React + Tailwind component testing + name: npm-react-tailwind path: examples/tailwind requires: - npm-react @@ -1785,7 +1793,7 @@ linux-workflow: &linux-workflow # requires: # - npm-react - npm-react-e2e-example: - name: React + Percy component testing + name: npm-react-percy path: examples/visual-testing-with-percy requires: - npm-react @@ -1796,36 +1804,23 @@ linux-workflow: &linux-workflow # requires: # - npm-react - npm-react-e2e-example: - name: React + Webpack file component testing + name: npm-react-webpack-file path: examples/webpack-file requires: - npm-react - npm-react-e2e-example: - name: React + Webpack options component testing + name: npm-react-webpack-options path: examples/webpack-options requires: - npm-react - - npm-react-release: - filters: - branches: - only: - - master - requires: - - build - - npm-vue - - npm-eslint-plugin-dev: requires: - build - - npm-eslint-plugin-dev-release: - filters: - branches: - only: - - master + + - npm-release: requires: - build - - npm-eslint-plugin-dev # various testing scenarios, like building full binary # and testing it on a real project diff --git a/npm/eslint-plugin-dev/package.json b/npm/eslint-plugin-dev/package.json index 2b1a8dd29b2a..9e91d78b4638 100644 --- a/npm/eslint-plugin-dev/package.json +++ b/npm/eslint-plugin-dev/package.json @@ -53,5 +53,8 @@ "cypress", "eslint", "eslintplugin" + ], + "ciJobs": [ + "npm-eslint-plugin-dev" ] } diff --git a/npm/react/package.json b/npm/react/package.json index 5bbab242b2b1..e987ac38ae59 100644 --- a/npm/react/package.json +++ b/npm/react/package.json @@ -159,6 +159,20 @@ "optional": true } }, + "ciJobs": [ + "npm-react", + "npm-react-axe", + "npm-react-next", + "npm-react-cra", + "npm-react-cra-folder", + "npm-react-rollup", + "npm-react-sass-ts", + "npm-react-snapshots", + "npm-react-tailwind", + "npm-react-percy", + "npm-react-webpack-file", + "npm-react-webpack-options" + ], "standard": { "globals": [ "Cypress", diff --git a/npm/vue/package.json b/npm/vue/package.json index 9cea65ab81a2..0adbc08fcfca 100644 --- a/npm/vue/package.json +++ b/npm/vue/package.json @@ -73,5 +73,8 @@ ], "publishConfig": { "registry": "http://registry.npmjs.org/" - } + }, + "ciJobs": [ + "npm-vue" + ] } diff --git a/npm/webpack-preprocessor/package.json b/npm/webpack-preprocessor/package.json index 9f91c731f5e9..c1486e9946a8 100644 --- a/npm/webpack-preprocessor/package.json +++ b/npm/webpack-preprocessor/package.json @@ -88,5 +88,12 @@ "cypress-plugin", "cypress-preprocessor", "webpack" + ], + "ciDependents": [ + "@cypress/react", + "@cypress/vue" + ], + "ciJobs": [ + "npm-webpack-preprocessor" ] } diff --git a/package.json b/package.json index 825e904bd110..b2fa31cd307b 100644 --- a/package.json +++ b/package.json @@ -14,6 +14,7 @@ "build": "lerna run build --stream", "build-prod": "lerna run build-prod --stream", "bump": "node ./scripts/binary.js bump", + "check-halt": "node scripts/check-halt.js", "check-next-dev-version": "node scripts/check-next-dev-version.js", "check-node-version": "node scripts/check-node-version.js", "check-terminal": "node scripts/check-terminal.js", @@ -41,6 +42,7 @@ "lint": "eslint --ext .js,.jsx,.ts,.tsx,.json .", "lint-changed": "lint-changed", "move-binaries": "node ./scripts/binary.js move-binaries", + "npm-release": "node scripts/npm-release.js", "semantic-release": "node ./scripts/semantic-release.js", "set-next-ci-version": "node ./scripts/binary.js setNextVersion", "prestart": "yarn ensure-deps", @@ -180,6 +182,7 @@ "ramda": "0.24.1", "semantic-release": "17.1.1", "semantic-release-monorepo": "7.0.3", + "semver": "7.3.2", "shelljs": "0.8.3", "shx": "0.3.2", "sinon": "7.3.2", diff --git a/patches/semantic-release-monorepo+7.0.3.patch b/patches/semantic-release-monorepo+7.0.3.patch deleted file mode 100644 index 343896a48b66..000000000000 --- a/patches/semantic-release-monorepo+7.0.3.patch +++ /dev/null @@ -1,142 +0,0 @@ -diff --git a/node_modules/semantic-release-monorepo/src/lerna-utils.js b/node_modules/semantic-release-monorepo/src/lerna-utils.js -new file mode 100644 -index 0000000..1b620f8 ---- /dev/null -+++ b/node_modules/semantic-release-monorepo/src/lerna-utils.js -@@ -0,0 +1,24 @@ -+const execa = require('execa'); -+ -+const lerna = async (args, options = {}) => { -+ const { stdout } = await execa('lerna', args, options); -+ return stdout; -+}; -+ -+/** -+ * @async -+ * @return {Promise} Registered lerna packages -+ */ -+const getPackageInfo = async () => JSON.parse(await lerna(['la', '--json'])); -+ -+/** -+ * @async -+ * @return {Promise} Registered lerna packages and their lerna dependencies -+ */ -+const getDependencyGraph = async () => -+ JSON.parse(await lerna(['la', '--graph'])); -+ -+module.exports = { -+ getPackageInfo, -+ getDependencyGraph, -+}; -diff --git a/node_modules/semantic-release-monorepo/src/only-package-commits.js b/node_modules/semantic-release-monorepo/src/only-package-commits.js -index fbcb315..1c39529 100644 ---- a/node_modules/semantic-release-monorepo/src/only-package-commits.js -+++ b/node_modules/semantic-release-monorepo/src/only-package-commits.js -@@ -1,10 +1,11 @@ --const { identity, memoizeWith, pipeP } = require('ramda'); -+const { identity, memoizeWith, pipeP, propEq } = require('ramda'); - const pkgUp = require('pkg-up'); - const readPkg = require('read-pkg'); - const path = require('path'); - const pLimit = require('p-limit'); - const debug = require('debug')('semantic-release:monorepo'); - const { getCommitFiles, getRoot } = require('./git-utils'); -+const { getPackageInfo, getDependencyGraph } = require('./lerna-utils'); - const { mapCommits } = require('./options-transforms'); - - const memoizedGetCommitFiles = memoizeWith(identity, getCommitFiles); -@@ -12,11 +13,50 @@ const memoizedGetCommitFiles = memoizeWith(identity, getCommitFiles); - /** - * Get the normalized PACKAGE root path, relative to the git PROJECT root. - */ -+const normalizedPath = async packagePath => { -+ const gitRoot = await getRoot(); -+ -+ return path.relative(gitRoot, packagePath); -+}; -+ - const getPackagePath = async () => { - const packagePath = await pkgUp(); -- const gitRoot = await getRoot(); - -- return path.relative(gitRoot, path.resolve(packagePath, '..')); -+ return normalizedPath(path.resolve(packagePath, '..')); -+}; -+ -+const getUsedLocalPrivatePackages = async () => { -+ const { dependencies } = await readPkg(); -+ -+ if (!dependencies) { -+ return []; -+ } -+ -+ const lernaPackages = await getPackageInfo(); -+ const privatePackages = lernaPackages -+ .filter(propEq('private', true)) -+ .filter((p) => !p.name.includes('@package') && !p.name === 'cypress'); -+ const dependencyGraph = await getDependencyGraph(); -+ -+ // build a list of private local packages -+ // including all children -+ const localPackages = []; -+ -+ const addLocalDependency = dep => { -+ const dependency = privatePackages.find(propEq('name', dep)); -+ -+ if (dependency && !localPackages.includes(dependency)) { -+ localPackages.push(dependency); -+ -+ const dependencies = dependencyGraph[dep]; -+ dependencies.forEach(addLocalDependency); -+ } -+ }; -+ -+ Object.keys(dependencies).forEach(addLocalDependency); -+ -+ // return paths to packages relative to git root -+ return Promise.all(localPackages.map(dep => normalizedPath(dep.location))); - }; - - const withFiles = async commits => { -@@ -33,21 +73,30 @@ const withFiles = async commits => { - - const onlyPackageCommits = async commits => { - const packagePath = await getPackagePath(); -- debug('Filter commits by package path: "%s"', packagePath); -+ debug('Package path: "%s"', packagePath); -+ -+ const localPackages = await getUsedLocalPrivatePackages(); -+ debug('Local packages: "%o"', localPackages); -+ -+ const paths = [packagePath, ...localPackages]; -+ debug('Filter commits by package paths: "%o"', paths); -+ - const commitsWithFiles = await withFiles(commits); -- // Convert package root path into segments - one for each folder -- const packageSegments = packagePath.split(path.sep); -+ -+ // Convert paths into segments - one for each folder -+ const pathSegments = paths.map(p => p.split(path.sep)); - - return commitsWithFiles.filter(({ files, subject }) => { - // Normalise paths and check if any changed files' path segments start -- // with that of the package root. -- const packageFile = files.find(file => { -- const fileSegments = path.normalize(file).split(path.sep); -- // Check the file is a *direct* descendent of the package folder (or the folder itself) -- return packageSegments.every( -- (packageSegment, i) => packageSegment === fileSegments[i] -- ); -- }); -+ // with that of the package root or local dependency paths -+ const packageFile = files.find(file => -+ pathSegments.some(segments => { -+ const fileSegments = path.normalize(file).split(path.sep); -+ -+ // Check the file is a *direct* descendent of the path -+ return segments.every((segment, i) => segment === fileSegments[i]); -+ }) -+ ); - - if (packageFile) { - debug( diff --git a/scripts/changed-packages.js b/scripts/changed-packages.js index cdc6b2034c01..38c37e854e11 100644 --- a/scripts/changed-packages.js +++ b/scripts/changed-packages.js @@ -1,52 +1,143 @@ +/* eslint-disable no-console */ const execa = require('execa') +const fs = require('fs') const path = require('path') // lists all packages that have changed from develop // and all packages that depend on those -const main = async () => { - const { stdout: root } = await execa('git', ['rev-parse', '--show-toplevel']) - const { stdout: diff } = await execa('git', ['merge-base', 'origin/develop', 'HEAD']) - const { stdout: filesChanged } = await execa('git', ['diff', '--name-only', diff]) - const { stdout: depGraph } = await execa('npx', ['lerna', 'la', '--graph']) - const { stdout: packs } = await execa('npx', ['lerna', 'la', '--json']) +// some files that are intimately related to the binary exist outside of lerna packages +// so we want to make sure they're considered to be part of the binary +const containsBinaryOutsideLerna = (changedFiles) => { + const binaryFiles = [ + '.node-version', + 'circle.yml', + 'electron-builder.json', + 'package.json', + 'yarn.lock', + ] - const files = filesChanged.split('\n') - const packages = JSON.parse(packs) - const dependencies = JSON.parse(depGraph) + return !!changedFiles.find((f) => f.includes('scripts/') || binaryFiles.includes(f)) +} - const findDependents = (packs) => { - const output = [...packs] +// merges `cypress` and all binary packages prefixed with `@packages/` +// into a single `cypress` output +const convertPackagesToBinary = (packages) => { + let output = [...packages] + let includeBinary = false - for (let d of Object.keys(dependencies)) { - if (!packs.includes(d) && packs.some((p) => dependencies[d].includes(p))) { - output.push(d) - } - } + output = output.filter((name) => { + const packageInBinary = name === 'cypress' || name.includes('@packages') + + includeBinary = includeBinary || packageInBinary + + return !packageInBinary + }) + + if (includeBinary) { + output.unshift('cypress') + } + + return output +} - return output.length === packs.length ? output : findDependents(output) +const getLernaPackages = async () => { + const { stdout } = await execa('npx', ['lerna', 'la', '--json']) + + return JSON.parse(stdout) +} + +// gets all files that have changed since the current branch diverged from some base branch +const getChangedFiles = async (base = 'origin/develop', output = false) => { + const { stdout: diff } = await execa('git', ['merge-base', base, 'HEAD']) + const { stdout: filesChanged } = await execa('git', ['diff', '--name-only', diff]) + + if (output) { + console.log(`Comparing against ${diff}`) + console.log(`Found the following changed files:`) + console.log(filesChanged) } + return filesChanged.split('\n') +} + +const getChangedPackages = async (base = 'origin/develop', output = false) => { + const { stdout: root } = await execa('git', ['rev-parse', '--show-toplevel']) + + const packages = await getLernaPackages() + const files = await getChangedFiles(base, output) + + // checks if a lerna package is changed const isChanged = ({ location }) => { const dir = path.relative(root, location) return !!files.find((f) => f.includes(dir)) } - const changed = [] + let changed = packages.filter(isChanged).map((p) => p.name) - for (let pack of packages) { - const dependents = findDependents([pack.name]) + changed = convertPackagesToBinary(changed) - for (let dep of dependents) { - if (!changed.includes(dep) && isChanged((packages.find((p) => p.name === dep)))) { - changed.push(dep) - } + if (!changed.includes('cypress') && containsBinaryOutsideLerna(files)) { + changed.unshift('cypress') + } + + if (output) { + console.log() + console.log(`The following packages were changed:`) + console.log(changed.join('\n')) + } + + return changed +} + +// finds dependents as defined in the `ciDependents` field +// within the package.json of the package - see CONTRIBUTING.md for docs +const getPackageDependents = async (name) => { + const packages = await getLernaPackages() + const pack = packages.find((p) => p.name === name) + + if (!pack) { + throw new Error('Could not find that package!') + } + + const packageJson = JSON.parse(fs.readFileSync(path.join(pack.location, 'package.json'))) + + return packageJson['ciDependents'] || [] +} + +// gets all of the changed packages and their corresponding dependents +const getChangedPackagesAndDependents = async (base = 'origin/develop', output = false) => { + const changedPackages = await getChangedPackages(base, output) + + const dependents = {} + + for (const pack of changedPackages) { + dependents[pack] = await getPackageDependents(pack) + } + + if (output) { + console.log() + console.log(`Changed packages and their dependents:`) + for (const pack in dependents) { + console.log(`${pack}: ${dependents[pack].join(', ') || 'none'}`) } } - /* eslint-disable-next-line no-console */ - console.log(changed.join('\n')) + return dependents } -main() +// execute main function if called from command line +if (require.main === module) { + const argv = require('minimist')(process.argv.slice(2)) + const base = argv._[0] + + getChangedPackagesAndDependents(base, true) +} + +module.exports = { + getLernaPackages, + getChangedPackages, + getPackageDependents, + getChangedPackagesAndDependents, +} diff --git a/scripts/check-halt.js b/scripts/check-halt.js new file mode 100644 index 000000000000..6dbe0917c385 --- /dev/null +++ b/scripts/check-halt.js @@ -0,0 +1,69 @@ +/* eslint-disable no-console */ +const { getCurrentBranch, readPackageJson } = require('./utils') +const { getChangedPackagesAndDependents, getLernaPackages } = require('./changed-packages') + +const runTestsAndExit = () => { + process.exit(0) +} + +const skipTestsAndExit = () => { + process.exit(1) +} + +const main = async (ciJob) => { + if (!ciJob) { + console.log(`Could not get current CI job`) + + return skipTestsAndExit() + } + + const currentBranch = await getCurrentBranch() + + if (currentBranch === 'develop' || currentBranch === 'master') { + console.log(`Currently on ${currentBranch} - all tests run`) + + return runTestsAndExit() + } + + const packages = await getLernaPackages() + const changed = await getChangedPackagesAndDependents() + + const packageInfo = packages + .filter((pack) => !pack.private && !pack.name.includes('@packages')) + .find((p) => { + const packageJson = readPackageJson(p) + + return packageJson.ciJobs && packageJson.ciJobs.includes(ciJob) + }) + + // default to binary if we don't find an independent package + const pack = packageInfo ? packageInfo.name : 'cypress' + + console.log(`Found package ${pack} that corresponds to current job ${ciJob}.`) + + if (Object.keys(changed).includes(pack)) { + console.log(`${pack} was directly changed, so tests run.`) + + return runTestsAndExit() + } + + const dependenciesChanged = [] + + for (const c in changed) { + if (changed[c].includes(pack)) { + dependenciesChanged.push(c) + } + } + + if (dependenciesChanged.length) { + console.log(`${pack} is listed as a dependant of ${dependenciesChanged.join(', ')}, so tests run.`) + + return runTestsAndExit() + } + + console.log(`${pack} is unchanged and not dependent on any changed packages, so tests do not run.`) + + return skipTestsAndExit() +} + +main(process.env.CIRCLE_JOB) diff --git a/scripts/inject-npm-version.js b/scripts/inject-npm-version.js deleted file mode 100644 index 50e6f87181ae..000000000000 --- a/scripts/inject-npm-version.js +++ /dev/null @@ -1,130 +0,0 @@ -const execa = require('execa') -const fs = require('fs') -const path = require('path') -const { groupBy } = require('lodash') -const debug = require('debug')('cypress:semantic-release') -const info = require('debug')('cypress:semantic-release:info') - -// updates a public package's package.json -// replaces any local dependencies that have a * version -// with the actual version of that dependency -// if that dependency is also going to be released from this commit -// it updates with the new version - -/* eslint-disable no-console */ - -const getPackages = async () => { - const { stdout: packages } = await execa('npx', ['lerna', 'la', '--json']) - - return JSON.parse(packages) -} - -const getBinaryVersion = async () => { - const { stdout: root } = await execa('git', ['rev-parse', '--show-toplevel']) - const rootPath = path.join(root, 'package.json') - const rootPackage = JSON.parse(fs.readFileSync(rootPath)) - - return rootPackage.version -} - -const parseSemanticReleaseOutput = (output) => { - debug(`Semantic Release Output ${output}`) - const currentVersion = (output.match(/associated with version (\d+\.\d+\.\d+-?\S*)/) || [])[1] - const nextVersion = (output.match(/The next release version is (\d+\.\d+\.\d+-?\S*)/) || [])[1] - - debug(`Current Version`, currentVersion) - debug(`Next Version`, nextVersion) - - return nextVersion || currentVersion -} - -const getPackageVersion = async (pack) => { - debug(`package`, pack) - const { stdout: semrel } = await execa('npx', ['lerna', 'exec', '--scope', pack, '--', 'npx', '--no-install', 'semantic-release', '--dry-run']) - - const version = parseSemanticReleaseOutput(semrel) - - if (!version) { - console.log(`ERROR`) - console.log(`Couldn't find a current or next version for ${pack}`) - - process.exit(1) - } - - return version -} - -const main = async (name) => { - if (!name) return - - debug(`Setting local npm packages to the correct version in package.json`) - - const packages = await getPackages() - const packagesByPrivacy = groupBy(packages, 'private') - - info(`Found these packages...`) - info(`Packages that can be independently released:`) - packagesByPrivacy.false && packagesByPrivacy.false.forEach((p) => info(`- ${p.name}`)) - info(`Packages that cannot be released:`) - packagesByPrivacy.true && packagesByPrivacy.true.forEach((p) => info(`- ${p.name}`)) - - const packageNames = packages.map((p) => p.name) - - const pack = packages.find((p) => p.name === name) - - if (!pack) { - console.log(`Couldn't find package ${name}`) - process.exit(1) - } - - const packagePath = path.join(pack.location, 'package.json') - const packageJson = JSON.parse(fs.readFileSync(packagePath)) - - if (!packageJson.dependencies) { - console.log(`No dependencies so we're done here`) - process.exit(0) - } - - // filter dependencies to only include local packages - const dependencies = Object.keys(packageJson.dependencies).filter((d) => packageNames.includes(d)) - - for (let dep of dependencies) { - process.stdout.write(`${dep}: `) - - let version - - if (dep === 'cypress') { - // Cypress binary gets handled differently than everything else - version = await getBinaryVersion() - } else { - if (packages.find((p) => p.name === dep).private) { - console.log(`ERROR`) - console.log(`Cannot add ${dep} as a dependency since it is private`) - process.exit(1) - } - - version = await getPackageVersion(dep) - } - - debug(version) - - packageJson.dependencies[dep] = version - } - - fs.writeFileSync(packagePath, JSON.stringify(packageJson, null, 2)) - - debug(`package.json updated!`) -} - -// execute main function if called from command line -if (require.main === module) { - const argv = require('minimist')(process.argv.slice(2)) - const name = argv._[0] - - main(name) -} - -module.exports = { - parseSemanticReleaseOutput, - main, -} diff --git a/scripts/npm-release.js b/scripts/npm-release.js new file mode 100644 index 000000000000..d18fbb898905 --- /dev/null +++ b/scripts/npm-release.js @@ -0,0 +1,223 @@ +/* eslint-disable no-console */ +const execa = require('execa') +const fs = require('fs') +const path = require('path') +const semverSortNewestFirst = require('semver/functions/rcompare') + +const { getCurrentBranch, getPackagePath, readPackageJson, minutes, independentTagRegex } = require('./utils') +const { getLernaPackages, getPackageDependents } = require('./changed-packages') +const { waitForJobToPass } = require('./wait-on-circle-jobs') + +const error = (message) => { + if (require.main === module) { + console.error(`\nERROR!`) + console.error(message) + + process.exit(1) + } else { + throw new Error(message) + } +} + +const getTags = async () => { + const { stdout } = await execa('git', ['tag', '--merged', await getCurrentBranch()]) + + return stdout.split('\n') +} + +const getBinaryVersion = async () => { + const { stdout: root } = await execa('git', ['rev-parse', '--show-toplevel']) + const rootPath = path.join(root, 'package.json') + const rootPackage = JSON.parse(fs.readFileSync(rootPath)) + + return rootPackage.version +} + +const parseSemanticReleaseOutput = (output) => { + const currentVersion = (output.match(/associated with version (\d+\.\d+\.\d+-?\S*)/) || [])[1] + const nextVersion = (output.match(/The next release version is (\d+\.\d+\.\d+-?\S*)/) || [])[1] + + return { + currentVersion, + nextVersion, + } +} + +// in addition to getting the next version that's going to be released +// this serves as a good double check that the release will work before we actually do it +const getNextVersion = async (name) => { + // we cannot use the semantic-release javascript api + // since it will break semantic-release-monorepo plugin + const { stdout } = await execa('npx', ['lerna', 'exec', '--scope', name, '--', 'npx', '--no-install', 'semantic-release', '--dry-run']) + + return parseSemanticReleaseOutput(stdout).nextVersion +} + +// we manually check the last version on this branch as opposed to what semantic-release says +// since semantic-release may be not be configured on the current branch for a package +const getCurrentVersion = async (name) => { + const tags = await getTags() + + const versions = tags + .map((tag) => (tag.match(independentTagRegex(name)) || [])[1]) + .filter((tag) => tag) + .sort(semverSortNewestFirst) + + return versions[0] +} + +const getPackageVersions = async (packages) => { + console.log(`Finding package versions...\n`) + + const binaryVersion = await getBinaryVersion() + + console.log(`Cypress binary: ${binaryVersion}`) + + const versions = { + cypress: { + currentVersion: binaryVersion, + nextVersion: undefined, + }, + } + + for (const name of packages) { + console.log(`\n${name}`) + + const currentVersion = await getCurrentVersion(name) + const nextVersion = await getNextVersion(name) + + console.log(`Current version: ${currentVersion || 'N/A'}`) + console.log(`Next version: ${nextVersion || 'N/A'}`) + + versions[name] = { + currentVersion, + nextVersion, + } + } + + return versions +} + +// updates a public package's package.json +// replaces any local dependencies that have a * version +// with the actual numbered version of that dependency +// if that dependency is also going to be released from this run +// it updates with the new version +const injectVersions = (packagesToRelease, versions, packages) => { + console.log('\nInjecting versions into package.json files...') + + for (const name of packagesToRelease) { + console.log(`\nUpdating package.json of ${name}`) + + const info = packages.find((p) => p.name === name) + const packageJson = readPackageJson(info) + + if (packageJson.dependencies) { + for (const dependency in packageJson.dependencies) { + if (packageJson.dependencies[dependency] === '*') { + const version = versions[dependency].nextVersion || versions[dependency].currentVersion + + if (!version) { + return error(`Could not inject a version for ${dependency} since it has no current or next version`) + } + + packageJson.dependencies[dependency] = version + + console.log(`\t${dependency}: ${version}`) + } + } + + fs.writeFileSync(getPackagePath(info), JSON.stringify(packageJson, null, 2)) + } + } +} + +// we want to wait on all tests to pass for the packages that we want to release +// even if they aren't related to a specific package +// since releasing some but not all could break the package numbers we injected +// failing/passing all also ensures that stuff doesn't get out of sync if the job is rerun +const waitOnTests = async (names, packageInfo) => { + const packages = names.concat(...await Promise.all(names.map(getPackageDependents))) + + const jobs = [...new Set([].concat(...packages.map((name) => { + const pkg = packageInfo.find((p) => p.name === name) + + return readPackageJson(pkg).ciJobs || [] + })))] + + console.log(`\nWaiting on the following CI jobs: ${jobs.join(', ')}`) + + return Promise.all(jobs.map((job) => { + waitForJobToPass(job) + .timeout(minutes(60)) + .then(() => { + console.log(`${job} passed`) + }).catch(() => { + error(`${job} failed - cannot release`) + }) + })).then(() => { + console.log(`\nAll CI jobs passed`) + }) +} + +const releasePackages = async (packages) => { + console.log(`\nReleasing packages`) + + // it would make sense to run each release simultaneously with something like Promise.all() + // however this can cause a race condition within git (git lock throws an error) + // so we run them one by one to avoid this + for (const name of packages) { + console.log(`\nReleasing ${name}...`) + const { stdout } = await execa('npx', ['lerna', 'exec', '--scope', name, '--', 'npx', '--no-install', 'semantic-release']) + + console.log(`Released ${name} successfully:`) + console.log(stdout) + } + + console.log(`\nAll packages released successfully`) +} + +// goes through the release process for all of our independent npm projects +const main = async () => { + if (!process.env.CIRCLECI) { + return error(`Cannot run release process outside of Circle CI`) + } + + if (process.env.CIRCLE_PULL_REQUEST) { + return console.log(`Release process cannot be run on a PR`) + } + + const packages = await getLernaPackages() + const publicPackages = packages + .filter((pkg) => !pkg.private && !pkg.name.includes('@packages')) + .map((pkg) => pkg.name) + + console.log(`Found the following public packages: ${publicPackages.join(', ')}\n`) + + const versions = await getPackageVersions(publicPackages) + const packagesToRelease = Object.keys(versions).filter((key) => versions[key].nextVersion) + + console.log(`\nFound a new release for the following packages: ${packagesToRelease.join(', ')}`) + + if (!packagesToRelease.length) { + return console.log(`\nThere are no packages to release!`) + } + + injectVersions(packagesToRelease, versions, packages) + + await waitOnTests(packagesToRelease, packages) + + await releasePackages(packagesToRelease) + + console.log(`\n\nRelease process completed successfully!`) +} + +// execute main function if called from command line +if (require.main === module) { + main() +} + +module.exports = { + parseSemanticReleaseOutput, + readPackageJson, +} diff --git a/scripts/semantic-release.js b/scripts/semantic-release.js deleted file mode 100644 index 4a016c223bff..000000000000 --- a/scripts/semantic-release.js +++ /dev/null @@ -1,17 +0,0 @@ -/* eslint-disable no-console */ - -const { execSync } = require('child_process') -const path = require('path') -const argv = require('minimist')(process.argv.slice(2)) - -function exec (command, args = {}) { - console.log(command) - execSync(command, { stdio: 'inherit', ...args }) -} - -const pack = argv._[0] - -console.log(`Running semantic release for ${pack}`) - -exec(`node ${path.join(__dirname, 'inject-npm-version.js')} ${pack}`) -exec(`lerna exec --scope ${pack} -- npx --no-install semantic-release`) diff --git a/scripts/unit/inject-npm-version-spec.js b/scripts/unit/npm-release-spec.js similarity index 74% rename from scripts/unit/inject-npm-version-spec.js rename to scripts/unit/npm-release-spec.js index aca9a1593333..ac24062f7290 100644 --- a/scripts/unit/inject-npm-version-spec.js +++ b/scripts/unit/npm-release-spec.js @@ -1,10 +1,23 @@ const la = require('lazy-ass') -const { parseSemanticReleaseOutput } = require('../inject-npm-version') +const { parseSemanticReleaseOutput } = require('../npm-release') + +const semanticReleasePullRequest = ` +[semantic-release] › ℹ Running semantic-release version 17.1.1 +[semantic-release] › ✔ Loaded plugin "verifyConditions" from "@semantic-release/changelog" +[semantic-release] › ✔ Loaded plugin "verifyConditions" from "@semantic-release/git" +[semantic-release] › ✔ Loaded plugin "verifyConditions" from "@semantic-release/npm" +[semantic-release] › ✔ Loaded plugin "prepare" from "@semantic-release/changelog" +[semantic-release] › ✔ Loaded plugin "prepare" from "@semantic-release/git" +[semantic-release] › ✔ Loaded plugin "prepare" from "@semantic-release/npm" +[semantic-release] › ✔ Loaded plugin "publish" from "@semantic-release/npm" +[semantic-release] › ✔ Loaded plugin "addChannel" from "@semantic-release/npm" +[semantic-release] › ℹ This run was triggered by a pull request and therefore a new version won't be published. +` const semanticReleaseNoUpdate = (version) => { return ` -[semantic-release] › ℹ Running semantic-release version 17.0.4 +[semantic-release] › ℹ Running semantic-release version 17.1.1 [semantic-release] › ✔ Loaded plugin "verifyConditions" from "@semantic-release/npm" [semantic-release] › ✔ Loaded plugin "prepare" from "@semantic-release/npm" [semantic-release] › ✔ Loaded plugin "publish" from "@semantic-release/npm" @@ -45,7 +58,7 @@ const semanticReleaseNoUpdate = (version) => { const semanticReleaseUpdate = (oldVersion, newVersion) => { return ` -[semantic-release] › ℹ Running semantic-release version 17.0.4 +[semantic-release] › ℹ Running semantic-release version 17.1.1 [semantic-release] › ✔ Loaded plugin "verifyConditions" from "@semantic-release/npm" [semantic-release] › ✔ Loaded plugin "prepare" from "@semantic-release/npm" [semantic-release] › ✔ Loaded plugin "publish" from "@semantic-release/npm" @@ -127,38 +140,49 @@ const semanticReleaseUpdate = (oldVersion, newVersion) => { ` } -describe('inject npm version', () => { +describe('semantic release', () => { + it('ends with no output if triggered by a pull request', () => { + const { currentVersion, nextVersion } = parseSemanticReleaseOutput(semanticReleasePullRequest) + + la(currentVersion === undefined, 'Expected current version to be', undefined, 'but got', currentVersion, 'instead') + la(nextVersion === undefined, 'Expected current version to be', undefined, 'but got', nextVersion, 'instead') + }) + describe('parses old version number when there are no updates', () => { it('works with standard version number', () => { const version = '1.2.3' - const result = parseSemanticReleaseOutput(semanticReleaseNoUpdate(version)) + const { currentVersion, nextVersion } = parseSemanticReleaseOutput(semanticReleaseNoUpdate(version)) - la(version === result, 'Expected version to be', version, 'but got', result, 'instead') + la(currentVersion === version, 'Expected current version to be', version, 'but got', currentVersion, 'instead') + la(nextVersion === undefined, 'Expected next version to be', version, 'but got', nextVersion, 'instead') }) it('works with version 0.x.x', () => { const version = '0.0.1' - const result = parseSemanticReleaseOutput(semanticReleaseNoUpdate(version)) + const { currentVersion, nextVersion } = parseSemanticReleaseOutput(semanticReleaseNoUpdate(version)) - la(version === result, 'Expected version to be', version, 'but got', result, 'instead') + la(currentVersion === version, 'Expected current version to be', version, 'but got', currentVersion, 'instead') + la(nextVersion === undefined, 'Expected next version to be', version, 'but got', nextVersion, 'instead') }) it('works with postfix alpha/beta version', () => { const version = '0.1.2-alpha1.2' - const result = parseSemanticReleaseOutput(semanticReleaseNoUpdate(version)) + const { currentVersion, nextVersion } = parseSemanticReleaseOutput(semanticReleaseNoUpdate(version)) - la(version === result, 'Expected version to be', version, 'but got', result, 'instead') + la(currentVersion === version, 'Expected current version to be', version, 'but got', currentVersion, 'instead') + la(nextVersion === undefined, 'Expected next version to be', version, 'but got', nextVersion, 'instead') }) it('does not work with non-semver version', () => { const version = 'abc' - const result = parseSemanticReleaseOutput(semanticReleaseNoUpdate(version)) + const { currentVersion, nextVersion } = parseSemanticReleaseOutput(semanticReleaseNoUpdate(version)) - la(typeof result === 'undefined', 'Expected version to be undefined but got', result, 'instead') + la(currentVersion === undefined, 'Expected current version to be', version, 'but got', currentVersion, 'instead') + la(nextVersion === undefined, 'Expected next version to be', version, 'but got', nextVersion, 'instead') }) }) @@ -167,50 +191,50 @@ describe('inject npm version', () => { const oldVersion = '1.2.3' const newVersion = '1.2.4' - const result = parseSemanticReleaseOutput(semanticReleaseUpdate(oldVersion, newVersion)) + const { currentVersion, nextVersion } = parseSemanticReleaseOutput(semanticReleaseUpdate(oldVersion, newVersion)) - la(oldVersion !== result, 'Expected version not to be', oldVersion, 'but got', result) - la(newVersion === result, 'Expected version to be', newVersion, 'but got', result, 'instead') + la(currentVersion === oldVersion, 'Expected current version to be', oldVersion, 'but got', currentVersion, 'instead') + la(nextVersion === newVersion, 'Expected next version to be', newVersion, 'but got', nextVersion, 'instead') }) it('works with 0.x.x version numbers', () => { const oldVersion = '0.0.1' const newVersion = '0.1.0' - const result = parseSemanticReleaseOutput(semanticReleaseUpdate(oldVersion, newVersion)) + const { currentVersion, nextVersion } = parseSemanticReleaseOutput(semanticReleaseUpdate(oldVersion, newVersion)) - la(oldVersion !== result, 'Expected version not to be', oldVersion, 'but got', result) - la(newVersion === result, 'Expected version to be', newVersion, 'but got', result, 'instead') + la(currentVersion === oldVersion, 'Expected current version to be', oldVersion, 'but got', currentVersion, 'instead') + la(nextVersion === newVersion, 'Expected next version to be', newVersion, 'but got', nextVersion, 'instead') }) it('works with 0.x.x -> 1.0.0 version numbers', () => { const oldVersion = '0.2.4' const newVersion = '1.0.0' - const result = parseSemanticReleaseOutput(semanticReleaseUpdate(oldVersion, newVersion)) + const { currentVersion, nextVersion } = parseSemanticReleaseOutput(semanticReleaseUpdate(oldVersion, newVersion)) - la(oldVersion !== result, 'Expected version not to be', oldVersion, 'but got', result) - la(newVersion === result, 'Expected version to be', newVersion, 'but got', result, 'instead') + la(currentVersion === oldVersion, 'Expected current version to be', oldVersion, 'but got', currentVersion, 'instead') + la(nextVersion === newVersion, 'Expected next version to be', newVersion, 'but got', nextVersion, 'instead') }) it('works with postfix alpha/beta versions', () => { const oldVersion = '0.2.4-alpha' const newVersion = '0.3.0-beta' - const result = parseSemanticReleaseOutput(semanticReleaseUpdate(oldVersion, newVersion)) + const { currentVersion, nextVersion } = parseSemanticReleaseOutput(semanticReleaseUpdate(oldVersion, newVersion)) - la(oldVersion !== result, 'Expected version not to be', oldVersion, 'but got', result) - la(newVersion === result, 'Expected version to be', newVersion, 'but got', result, 'instead') + la(currentVersion === oldVersion, 'Expected current version to be', oldVersion, 'but got', currentVersion, 'instead') + la(nextVersion === newVersion, 'Expected next version to be', newVersion, 'but got', nextVersion, 'instead') }) it('works with postfix alpha/beta version -> 1.0.0', () => { const oldVersion = '0.2.4-alpha' const newVersion = '1.0.0' - const result = parseSemanticReleaseOutput(semanticReleaseUpdate(oldVersion, newVersion)) + const { currentVersion, nextVersion } = parseSemanticReleaseOutput(semanticReleaseUpdate(oldVersion, newVersion)) - la(oldVersion !== result, 'Expected version not to be', oldVersion, 'but got', result) - la(newVersion === result, 'Expected version to be', newVersion, 'but got', result, 'instead') + la(currentVersion === oldVersion, 'Expected current version to be', oldVersion, 'but got', currentVersion, 'instead') + la(nextVersion === newVersion, 'Expected next version to be', newVersion, 'but got', nextVersion, 'instead') }) }) }) diff --git a/scripts/utils.js b/scripts/utils.js index 048f825b444b..4f4294eebf0a 100644 --- a/scripts/utils.js +++ b/scripts/utils.js @@ -3,6 +3,7 @@ const la = require('lazy-ass') const is = require('check-more-types') const path = require('path') const fs = require('fs') +const execa = require('execa') /* eslint-disable no-console */ @@ -113,10 +114,31 @@ const getCIBuildUrl = () => { } } +const seconds = (s) => s * 1000 +const minutes = (m) => m * 60 * 1000 + +const getCurrentBranch = async () => { + const { stdout } = await execa('git', ['rev-parse', '--abbrev-ref', 'HEAD']) + + return stdout +} + +const getPackagePath = ({ location }) => path.join(location, 'package.json') + +const readPackageJson = (pack) => JSON.parse(fs.readFileSync(getPackagePath(pack))) + +const independentTagRegex = (name) => new RegExp(`^${name}-v(.+)`) + module.exports = { getNameAndBinary, getJustVersion, getShortCommit, getCIName, getCIBuildUrl, + seconds, + minutes, + getCurrentBranch, + getPackagePath, + readPackageJson, + independentTagRegex, } diff --git a/scripts/wait-on-circle-jobs.js b/scripts/wait-on-circle-jobs.js index 09eee4afb5b4..25fd973d21cc 100644 --- a/scripts/wait-on-circle-jobs.js +++ b/scripts/wait-on-circle-jobs.js @@ -8,6 +8,8 @@ const got = require('got') // always print the debug logs const debug = require('debug')('*') +const { seconds, minutes } = require('./utils') + // we expect CircleCI to set the current polling job name const jobName = process.env.CIRCLE_JOB || 'wait-on-circle-jobs' @@ -15,34 +17,18 @@ const workflowId = process.env.CIRCLE_WORKFLOW_ID const getAuth = () => `${process.env.CIRCLE_TOKEN}:` -if (!process.env.CIRCLE_TOKEN) { - console.error('Cannot find CIRCLE_TOKEN') - process.exit(1) -} - -if (!process.env.CIRCLE_WORKFLOW_ID) { - console.error('Cannot find CIRCLE_WORKFLOW_ID') - process.exit(1) -} - -const args = minimist(process.argv.slice(2), { boolean: false }) - -const jobNames = _ -.chain(args['job-names']) -.split(',') -.without('true') -.map(_.trim) -.compact() -.value() +const verifyCI = () => { + if (!process.env.CIRCLE_TOKEN) { + console.error('Cannot find CIRCLE_TOKEN') + process.exit(1) + } -if (!jobNames.length) { - console.error('Missing argument: --job-names') - console.error('You must pass a comma separated list of Circle CI job names to wait for.') - process.exit(1) + if (!process.env.CIRCLE_WORKFLOW_ID) { + console.error('Cannot find CIRCLE_WORKFLOW_ID') + process.exit(1) + } } -debug('received circle jobs: %o', jobNames) - /* eslint-disable-next-line no-unused-vars */ const getWorkflow = async (workflowId) => { const auth = getAuth() @@ -71,7 +57,7 @@ const getWorkflow = async (workflowId) => { * - running (currently running) * - failed | success */ -const getJobStatus = async (workfowId) => { +const getJobStatus = async (workflowId) => { const auth = getAuth() // typo at https://circleci.com/docs/2.0/api-intro/ // to retrieve all jobs, the url is "/workflow/:id/job" @@ -98,7 +84,7 @@ const getJobStatus = async (workfowId) => { return response } -const waitForAllJobs = async (workflowId) => { +const waitForAllJobs = async (jobNames, workflowId) => { let response try { @@ -141,27 +127,89 @@ const waitForAllJobs = async (workflowId) => { return Promise.reject(new Error('Jobs have not finished')) } -// finished, has one failed job -// const workflowId = '566ffe9a-62d4-45cd-9a27-9882139e0121' -// pending workflow -// jobs that have not run have "status: 'blocked'" - -// getWorkflow(workflowId).then(console.log, console.error) -// getWorkflowJobs(workflowId).then(console.log, console.error) - -const seconds = (s) => s * 1000 -const minutes = (m) => m * 60 * 1000 - -// https://github.com/demmer/bluebird-retry -retry(waitForAllJobs.bind(null, workflowId), { - timeout: minutes(30), // max time for this job - interval: seconds(30), // poll intervals - max_interval: seconds(30), -}).then(() => { - console.log('all done') -}, (err) => { - console.error(err) - process.exit(1) -}) - -// getJobStatus(workflowId).then(console.log, console.error) +const waitForJobToPass = async (jobName, workflow = workflowId) => { + verifyCI() + + let response + + try { + response = await getJobStatus(workflow) + } catch (e) { + console.error(e) + process.exit(1) + } + + const job = _.find(response.items, { name: jobName }) + + if (!job) { + return Promise.reject(new Error('Job not found')) + } + + const { status } = job + + if (status === 'success') { + return Promise.resolve() + } + + if (status === 'failed') { + return Promise.reject(new Error('Job failed')) + } + + await Promise.delay(seconds(30)) + + return waitForJobToPass(jobName, workflow) +} + +const main = () => { + verifyCI() + + const args = minimist(process.argv.slice(2), { boolean: false }) + + const jobNames = _ + .chain(args['job-names']) + .split(',') + .without('true') + .map(_.trim) + .compact() + .value() + + if (!jobNames.length) { + console.error('Missing argument: --job-names') + console.error('You must pass a comma separated list of Circle CI job names to wait for.') + process.exit(1) + } + + debug('received circle jobs: %o', jobNames) + + // finished, has one failed job + // const workflowId = '566ffe9a-62d4-45cd-9a27-9882139e0121' + // pending workflow + // jobs that have not run have "status: 'blocked'" + + // getWorkflow(workflowId).then(console.log, console.error) + // getWorkflowJobs(workflowId).then(console.log, console.error) + + // https://github.com/demmer/bluebird-retry + retry(waitForAllJobs.bind(null, jobNames, workflowId), { + timeout: minutes(30), // max time for this job + interval: seconds(30), // poll intervals + max_interval: seconds(30), + }).then(() => { + console.log('all done') + }, (err) => { + console.error(err) + process.exit(1) + }) + + // getJobStatus(workflowId).then(console.log, console.error) +} + +// execute main function if called from command line +if (require.main === module) { + main() +} + +module.exports = { + minutes, + waitForJobToPass, +}