Bots cleanup, avoid leaving inline reviews when N>5 (facebook#24923)

Summary:
This PR cleans up some of our GitHub bots. The overall goal is to make the contribution process just a tad nicer.

### analysis-bot

* The bot will continue leaving GitHub Reviews when it finds lint issues, but will abstain from leaving inline comments if they would exceed 5 in number.
* The review comment left by the bot has instructions on how to reproduce the lint issues locally. This will educate PR authors on how to run lint and fix the issues without unnecessarily spamming the PR with 50+ comments, while still providing useful reviews to authors when only a handful of lint issues slip by.
* Code moved to `bots/` directory for ease of discovery and co-location with pull-bot.
* Added `yarn lint-ci` command. This seems like the right choice: it's running `yarn lint` and other linters, and it is only intended to run on CI.
* It's still possible to run `yarn lint-ci` locally, though the script will stop short of posting a review to GitHub unless the necessary envvars are provided.
* Added `yarn shellcheck` command. This can be run locally, though it requires `shellcheck` to be installed.
* Outside of this PR, I added instructions on using shellcheck to https://github.com/facebook/react-native/wiki/Development-Dependencies
* Updated Circle CI config to use these new commands, and streamlined the `analyze_pr` step.
* Documented analysis-bot in `bots/README.md`.

### pull-bot

* Bumped `danger-js` dependency. No breaking changes found in this minor bump from what I can tell.
* Documented pull-bot in `bots/README.md`.

### misc

* PR template: don't use jargon.

## Changelog

[Internal] [Changed] - GitHub Bots cleanup
Pull Request resolved: facebook#24923

Differential Revision: D15399744

Pulled By: hramos

fbshipit-source-id: 32632e775f8554424072270e3f98542de84bfb8c

.circleci/config.yml

@@ -215,44 +215,27 @@ jobs:
 
       - restore-cache: *restore-yarn-cache
       - run: *yarn
-
       - run:
-          name: Analyze Shell Scripts
+          name: Install dependencies
           command: |
-            echo -e "\\x1B[36mInstalling additional dependencies\\x1B[0m"
-            apt update && apt install -y shellcheck jq
-            yarn add @octokit/[email protected]
-            echo -e "\\x1B[36mAnalyzing shell scripts\\x1B[0m"; \
-            GITHUB_TOKEN="$PUBLIC_ANALYSISBOT_GITHUB_TOKEN_A""$PUBLIC_ANALYSISBOT_GITHUB_TOKEN_B" \
-            GITHUB_OWNER="$CIRCLE_PROJECT_USERNAME" \
-            GITHUB_REPO="$CIRCLE_PROJECT_REPONAME" \
-            GITHUB_PR_NUMBER="$CIRCLE_PR_NUMBER" \
-            ./scripts/circleci/analyze_scripts.sh
-          when: always
+            apt update
+            apt install -y shellcheck jq
+            cd bots
+            yarn install --non-interactive --cache-folder ~/.cache/yarn
+      - save-cache: *save-yarn-cache
 
       - run:
-          name: Analyze Code
-          command: |
-            echo -e "\\x1B[36mInstalling additional dependencies\\x1B[0m"; yarn add @octokit/[email protected]
-            echo -e "\\x1B[36mAnalyzing code\\x1B[0m"; \
-            GITHUB_TOKEN="$PUBLIC_ANALYSISBOT_GITHUB_TOKEN_A""$PUBLIC_ANALYSISBOT_GITHUB_TOKEN_B" \
-            GITHUB_OWNER="$CIRCLE_PROJECT_USERNAME" \
-            GITHUB_REPO="$CIRCLE_PROJECT_REPONAME" \
-            GITHUB_PR_NUMBER="$CIRCLE_PR_NUMBER" \
-            ./scripts/circleci/analyze_code.sh
+          name: Run linters against modified files (analysis-bot)
+          command: GITHUB_TOKEN="$PUBLIC_ANALYSISBOT_GITHUB_TOKEN_A""$PUBLIC_ANALYSISBOT_GITHUB_TOKEN_B" yarn lint-ci
           when: always
 
       - run:
-          name: Analyze Pull Request
+          name: Analyze Pull Request (pull-bot)
           command: |
-            echo -e "\\x1B[36mInstalling additional dependencies\\x1B[0m"
             cd bots
-            yarn install --non-interactive --cache-folder ~/.cache/yarn
-            echo -e "\\x1B[36mAnalyzing pull request\\x1B[0m"; \
-            DANGER_GITHUB_API_TOKEN="$PUBLIC_PULLBOT_GITHUB_TOKEN_A""$PUBLIC_PULLBOT_GITHUB_TOKEN_B" \
-            yarn danger ci --use-github-checks
+            DANGER_GITHUB_API_TOKEN="$PUBLIC_PULLBOT_GITHUB_TOKEN_A""$PUBLIC_PULLBOT_GITHUB_TOKEN_B" yarn danger ci --use-github-checks
           when: always
-      - save-cache: *save-yarn-cache
+
 
   # -------------------------
   #    JOBS: Analyze Code

.github/PULL_REQUEST_TEMPLATE.md

@@ -12,4 +12,4 @@
 
 ## Test Plan
 
-<!-- Demonstrate the code is solid. Example: The exact commands you ran and their output, screenshots / videos if the pull request changes UI. -->
+<!-- Demonstrate the code is solid. Example: The exact commands you ran and their output, screenshots / videos if the pull request changes the user interface. -->

bots/README.md

@@ -1,18 +1,23 @@
-GitHub bots, such as the Danger bot, as well as messages used by the Facebook GitHub bot are all configured in this directory/
-
 ## Danger
 
-We use [Danger JS](http://danger.systems/js/) to perform rudimentary maintenance on the React Native repository. 
+[Danger](http://danger.systems/js/) is a JavaScript runtime which helps you provide continuous feedback inside GitHub. It's used by @pull-bot to analyze the contents of a GitHub pull request.
 
-If you'd like to make changes to the Dangerfile, find an existing PR on the React Native repo and make note of the URL.		
-		
-Then, run from the React Native root directory:
+If you want to test changes to Danger, I'd recommend checking out an existing PR and then running the `danger pr` command.
+You'll need a GitHub token. You can re-use this one: `a6edf8e8d40ce4e8b11a 150e1341f4dd9c944d2a` (just remove the space).
+So, for example:
 
 ```
-cd bots
-npm install
-..
-node bots/node_modules/.bin/danger pr https://github.com/facebook/react-native/pull/1		
+DANGER_GITHUB_API_TOKEN=[ENV_ABOVE] yarn danger pr https://github.com/facebook/react-native/pull/1234
 ```
 
-And you will get the responses from parsing the Dangerfile.
+## Code Analysis Bot
+
+The code analysis bot provides lint and other results as inline reviews on GitHub. It runs as part of the Circle CI analysis workflow.
+
+If you want to test changes to the Code Analysis Bot, I'd recommend checking out an existing PR and then running the `analyze pr` command.
+You'll need a GitHub token. You can re-use this one: `78a72af35445ca3f8180` `b1a98e0bbd56ff1ccba1` (just remove the space).
+So, for example:
+
+```
+GITHUB_TOKEN=[ENV_ABOVE] GITHUB_PR_NUMBER=1234 yarn lint-ci
+```

scripts/circleci/code-analysis-bot.js → bots/code-analysis-bot.js

@@ -33,8 +33,10 @@ function push(arr, key, value) {
 const converterSummary = {
   eslint:
     '`eslint` found some issues. Run `yarn lint --fix` to automatically fix problems.',
-  flow: '`flow` found some issues.',
-  shellcheck: '`shellcheck` found some issues.',
+  flow:
+    '`flow` found some issues. Run `yarn flow check` to analyze your code and address any errors.',
+  shellcheck:
+    '`shellcheck` found some issues. Run `yarn shellcheck` to analyze shell scripts.',
 };
 
 /**
@@ -172,6 +174,9 @@ function sendReview(owner, repo, number, commit_id, body, comments) {
     if (comments.length === 0) {
       // Do not leave an empty review.
       return;
+    } else if (comments.length > 5) {
+      // Avoid noisy reviews and rely solely on the body of the review.
+      comments = [];
     }
 
     const event = 'REQUEST_CHANGES';
@@ -227,7 +232,7 @@ function main(messages, owner, repo, number) {
     });
   } else {
     console.log(
-      'Missing GITHUB_TOKEN. Example: 5fd88b964fa214c4be2b144dc5af5d486a2f8c1e. Review feedback with code analysis results will not be provided on GitHub.',
+      'Missing GITHUB_TOKEN. Example: 5fd88b964fa214c4be2b144dc5af5d486a2f8c1e. Review feedback with code analysis results will not be provided on GitHub without a valid token.',
     );
   }
 
@@ -319,7 +324,9 @@ process.stdin.on('end', function() {
   const repo = process.env.GITHUB_REPO;
 
   if (!process.env.GITHUB_PR_NUMBER) {
-    console.error('Missing GITHUB_PR_NUMBER. Example: 4687');
+    console.error(
+      'Missing GITHUB_PR_NUMBER. Example: 4687. Review feedback with code analysis results cannot be provided on GitHub without a valid pull request number.',
+    );
     // for master branch, don't throw an error
     process.exit(0);
   }

bots/dangerfile.js

@@ -4,17 +4,13 @@
  * This source code is licensed under the MIT license found in the
  * LICENSE file in the root directory of this source tree.
  *
- * To test Danger during development, run yarn in this directory, then run:
- * $ yarn danger pr <URL to GitHub PR>
- *
  * @format
  */
 
 'use strict';
 
-const includes = require('lodash.includes');
-
 const {danger, fail, message, warn} = require('danger');
+const includes = require('lodash.includes');
 
 // Provides advice if a summary section is missing, or body is too short
 const includesSummary =

bots/package.json

@@ -4,8 +4,11 @@
     "danger": "node ./node_modules/.bin/danger"
   },
   "devDependencies": {
-    "danger": "^7.0.8",
+    "danger": "^7.1.4",
     "lodash.includes": "^4.3.0",
     "minimatch": "^3.0.4"
+  },
+  "dependencies": {
+    "@octokit/rest": "15.18.0"
   }
 }