[Issue 314] 30k deliverable reporting

.github/workflows/ci-analytics.yml

@@ -0,0 +1,35 @@
+name: Analytics Checks
+
+on:
+  workflow_call:
+  pull_request:
+    paths:
+      - analytics/**
+      - .github/workflows/ci-analytics.yml
+
+defaults:
+  run:
+    working-directory: ./analytics
+
+jobs:
+  lint-test:
+    name: Analytics Lint, Format & Tests
+    runs-on: ubuntu-latest
+    steps:
+      # set up python
+      - uses: actions/checkout@v3
+      - uses: actions/setup-python@v4
+        with:
+          python-version: "3.11"
+
+      # install poetry
+      - uses: Gr1N/setup-poetry@v8
+
+      - name: Install analytics package using poetry
+        run: make install
+
+      - name: Run linting
+        run: make lint
+
+      - name: Run tests
+        run: make test

.github/workflows/run-analytics.yml

@@ -0,0 +1,33 @@
+# TODO(@widal001): 2023-11-20 - Update this file to run on a schedule: https://github.com/HHS/simpler-grants-gov/issues/742
+name: Run analytics package
+
+on:
+  workflow_dispatch:
+  pull_request:
+    paths:
+      - analytics/**
+      - .github/workflows/run-analytics.yml
+
+defaults:
+  run:
+    working-directory: ./analytics # ensures that this job runs from the analytics sub-directory
+
+jobs:
+  check-cli-help:
+    name: Check that the CLI is installed correctly
+    runs-on: ubuntu-latest
+    steps:
+      # set up python
+      - uses: actions/checkout@v3
+      - uses: actions/setup-python@v4
+        with:
+          python-version: "3.11"
+
+      # install poetry
+      - uses: Gr1N/setup-poetry@v8
+
+      - name: Install analytics package using poetry
+        run: make install
+
+      - name: Run CLI help to test that it's working
+        run: poetry run analytics --help

analytics/.gitignore

@@ -0,0 +1,4 @@
+data
+
+# Ignore dynaconf secret files
+.secrets.*

analytics/Makefile

@@ -1,7 +1,31 @@
+POETRY ?= poetry
+GITHUB ?= gh
+
+check-prereqs:
+	@echo "=> Checking for pre-requisites"
+	@if ! $(POETRY) --version; then echo "=> Poetry isn't installed"; fi
+	@if ! $(GITHUB) --version; then echo "=> GitHub CLI isn't installed"; fi
+	@echo "=> All pre-requisites satisfied"
+
+install: check-prereqs
+	@echo "=> Installing python dependencies"
+	$(POETRY) install
+
 setup:
-	poetry install
-	gh auth login
+	$(MAKE) install
+	$(GITHUB) auth login
+
+lint:
+	@echo "=> Running code quality checks"
+	@echo "============================="
+	$(POETRY) run black src tests
+	$(POETRY) run ruff src tests --fix
+	$(POETRY) run pylint src tests
+	$(POETRY) run mypy src
+	@echo "============================="
+	@echo "=> All checks succeeded"
 
-sprint_report:
-	poetry run python cli.py
-	poetry run jupyter notebook
+test:
+	@echo "=> Running tests"
+	@echo "============================="
+	$(POETRY) run pytest

analytics/README.md

@@ -6,36 +6,113 @@ This sub-directory enables users to run analytics on data generated within the S
 
 ### Pre-requisites
 
-- Python version 3.10 or 3.11
+- Python version 3.11
 - Poetry
 - GitHub CLI
 
-Check that you have the following with:
-
-```
-python --version
-poetry --version
-gh --version
-```
+Check that you have the following with: `make check-prereqs`
 
 ### Installation
 
 1. Clone the GitHub repo: `git clone https://github.com/HHS/simpler-grants-gov.git`
 2. Change directory into the analytics folder: `cd simpler-grants-gov/analytics`
-3. Check that you have the pre-requisites installed:
-   ```
-   python --version
-   poetry --version
-   gh --version
-   ```
 4. Set up the project: `make setup` -- This will install the required packages and prompt you to authenticate with GitHub
+5. Create a `.secrets.toml` with the following details:
+   ```toml
+   reporting_channel_id = "<REPLACE_WITH_CHANNEL_ID>"
+   slack_bot_token = "<REPLACE_WITH_SLACKBOT_TOKEN_ID>"
+   ```
+
+## Getting started
+
+### Learning how to use the command line tool
+
+The `analytics` package comes with a built-in CLI that you can use to discover the reporting features available:
+
+Start by simply typing `poetry run analytics --help` which will print out a list of available commands:
+
+![Screenshot of passing the --help flag to CLI entry point](static/screenshot-cli-help.png)
+
+Discover the arguments required for a particular command by appending the `--help` flag to that command:
+
+```bash
+poetry run analytics export gh_issue_data --help
+```
+
+![Screenshot of passing the --help flag to a specific command](static/screenshot-command-help.png)
+
+### Exporting GitHub data
+
+After following the installation steps above, you can use the following commands to export data from GitHub for local analysis:
+
+#### Exporting issue data
+
+```bash
+poetry run analytics export gh_issue_data --owner HHS --repo simpler-grants-gov --output-file data/issue-data.json
+```
+
+Let's break this down piece by piece:
+
+- `poetry run` - Tells poetry to execute a package installed in the virtual environment
+- `analytics` - The name of the analytics package installed locally
+- `export gh_issue_data` - The specific sub-command in the analytics CLI we want to run
+- `--owner HHS` Passing `HHS` to the `--owner` argument for this sub-command, the owner of the repo whose issue data we want to export, in this case `HHS`
+- `--repo simpler-grants-gov` We want to export issue data from the `simpler-grants-gov` repo owned by `HHS`
+- `--output-file data/issue-data.json` We want to write the exported data to the file with the relative path `data/issue-data.json`
 
-## Calculating Analytics
+#### Exporting project data
 
-### Running the Sprint Report
+Exporting project data works almost the same way, except it expects a `--project` argument instead of a `--repo` argument. **NOTE:** The project should be the project number as it appears in the URL, not the name of the project.
 
-From within the analytics sub-directory run: `make sprint_report`
+```bash
+poetry run analytics export gh_project_data --owner HHS --project 13 --output-file data/sprint-data.json
+```
+
+### Calculating metrics
+
+#### Calculating sprint burndown
+
+Once you've exported the sprint and issue data from GitHub, you can start calculating metrics. We'll begin with sprint burndown:
+
+```bash
+poetry run analytics calculate sprint_burndown --sprint-file data/sprint-data.json --issue-file data/issue-data.json --sprint @current --show-results
+```
+
+A couple of important notes about this command:
+
+- `--sprint @current` In order to calculate burndown, you'll need to specify either `@current` for the current sprint or the name of another sprint, e.g. `"Sprint 10"`
+- `--show-results` In order to the see the output in a browser you'll need to pass this flag.
+
+![Screenshot of burndown for sprint 10](static/reporting-notebook-screenshot.png)
+
+You can also post the results of this metric to a Slack channel:
+
+```bash
+poetry run analytics calculate sprint_burndown --sprint-file data/sprint-data.json --issue-file data/issue-data.json --sprint "Sprint 10" --post-results
+```
+
+> **NOTE:** This requires you to have the `.secrets.toml` configured according to the directions in step 5 of the [installation section](#installation)
+
+![Screenshot of burndown report in slack](static/screenshot-slack-burndown.png)
+
+### Calculating deliverable percent complete
+
+Another key metric you can report is the percentage of tasks or points completed per 30k deliverable.
+You can specify the unit you want to use for percent complete (e.g. points or tasks) using the `--unit` flag.
+
+For example, here we're calculating percentage completion based on the number of tickets under each deliverable.
+
+```bash
+poetry run analytics calculate deliverable_percent_complete --sprint-file data/sprint-data.json --issue-file data/issue-data.json --show-results --unit tasks
+```
+![Screenshot of deliverable percent complete by tasks](static/screenshot-deliverable-pct-complete-tasks.png)
+
+And here we're calculating it based on the total story point value of those tickets.
+
+```bash
+poetry run analytics calculate deliverable_percent_complete --sprint-file data/sprint-data.json --issue-file data/issue-data.json --show-results --unit points
+```
 
-This should open a new browser tab with a jupyter notebook see screenshot below:
+![Screenshot of deliverable percent complete by tasks](static/screenshot-deliverable-pct-complete-points.png)
 
-![Screenshot of jupyter notebook](static/reporting-notebook-screenshot.png)
+The `deliverable_pct_complete` sub-command also supports the `--post-results` flag if you want to post this data to slack.

analytics/config.py

@@ -0,0 +1,32 @@
+"""Loads configuration variables from settings files and settings files
+
+Dynaconf provides a few valuable features for configuration management:
+- Load variables from env vars and files with predictable overrides
+- Validate the existence and format of required configs
+- Connect with secrets managers like HashiCorp's Vault server
+- Load different configs based on environment (e.g. DEV, PROD, STAGING)
+
+For more information visit: https://www.dynaconf.com/
+"""
+from dynaconf import Dynaconf, Validator, ValidationError
+
+settings = Dynaconf(
+    # set env vars with `export ANALYTICS_FOO=bar`
+    envvar_prefix="ANALYTICS",
+    # looks for config vars in the following files
+    # with vars in .secrets.toml overriding vars in settings.toml
+    settings_files=["settings.toml", ".secrets.toml"],
+    # add validators for our required config vars
+    validators=[
+        Validator("SLACK_BOT_TOKEN", must_exist=True),
+        Validator("REPORTING_CHANNEL_ID", must_exist=True),
+    ],
+)
+
+# raises after all possible errors are evaluated
+try:
+    settings.validators.validate_all()
+except ValidationError as error:
+    list_of_all_errors = error.details
+    print(list_of_all_errors)
+    raise