Kibana Deployment

[elipe17]

Summary of Changes

• Added manifests to deploy Kibana and the Elastic proxy
• Updated Nginx to act as an auth endpoint for Kibana
• Removed elastic_setup container because it is now irrelevant
• Updated deployment configs
  Pull request closes Kibana Deployment #2790

How to Test

List the steps to test the PR
These steps are generic, please adjust as necessary.

cd tdrs-frontend && docker-compose up
cd tdrs-backend && docker-compose up

1. Open http://localhost:3000/ and sign in.
2. Navigate to /kibana and verify you get an error from nginx
3. Add BYPASS_KIBANA_AUTH=yes to your .env
4. Navigate to /kibana and verify you can see Kibana and it feels native to frontend
5. Navigate to https://tdp-frontend-raft.app.cloud.gov and perform the previous steps

• To see Kibana in the deployed environment follow the steps below to give yourself access. Make sure you have already requested access by way of the frontend. If you already have access, the steps below still apply. If you don't have the cf cli tool, let me know and I can perform the steps for you after you have requested access from the frontend.

1. Login to cloud foundry with the CLI: cf login --sso
2. Ssh into backend: cf ssh tdp-backend-raft
3. Enable python: /tmp/lifecycle/shell
4. Start Django shell: python manage.py shell_plus
5. Get your user: user = User.objects.all().filter(email="YOUR_LOGIN_EMAIL_HERE")[0]
6. Approve your user: user.account_approval_status = "Approved"
7. Get correct group: group = Group.objects.get(name="OFA System Admin")
8. Add yourself to group: group.user_set.add(user)
9. Give user a fake hhs_id: user.hhs_id = "111111111111"
10. Save: user.save()
11. Verify you can see the frontend now and the Kibana tab.

Deliverables

More details on how deliverables herein are assessed included here.

Deliverable 1: Accepted Features

Checklist of ACs:

• Kibana deploys to dev environment
• Routes between Kibana and Elastic are configured
• Kibana can connect to Elastic
• User can navigate to Kibana from frontend
• Testing Checklist has been run and all tests pass
• README is updated, if necessary

• lfrohlich and/or adpennington confirmed that ACs are met.

Deliverable 2: Tested Code

• Are all areas of code introduced in this PR meaningfully tested?
  • If this PR introduces backend code changes, are they meaningfully tested?
  • If this PR introduces frontend code changes, are they meaningfully tested?
• Are code coverage minimums met?
  • Frontend coverage: [insert coverage %] (see CodeCov Report comment in PR)
  • Backend coverage: [insert coverage %] (see CodeCov Report comment in PR)

Deliverable 3: Properly Styled Code

• Are backend code style checks passing on CircleCI?
• Are frontend code style checks passing on CircleCI?
• Are code maintainability principles being followed?

Deliverable 4: Accessible

• Does this PR complete the epic?
• Are links included to any other gov-approved PRs associated with epic?
• Does PR include documentation for Raft's a11y review?
• Did automated and manual testing with iamjolly and ttran-hub using Accessibility Insights reveal any errors introduced in this PR?

Deliverable 5: Deployed

• Was the code successfully deployed via automated CircleCI process to development on Cloud.gov?

Deliverable 6: Documented

• Does this PR provide background for why coding decisions were made?
• If this PR introduces backend code, is that code easy to understand and sufficiently documented, both inline and overall?
• If this PR introduces frontend code, is that code easy to understand and sufficiently documented, both inline and overall?
• If this PR introduces dependencies, are their licenses documented?
• Can reviewer explain and take ownership of these elements presented in this code review?

Deliverable 7: Secure

• Does the OWASP Scan pass on CircleCI?
• Do manual code review and manual testing detect any new security issues?
• If new issues detected, is investigation and/or remediation plan documented?

Deliverable 8: User Research

Research product(s) clearly articulate(s):

• the purpose of the research
• methods used to conduct the research
• who participated in the research
• what was tested and how
• impact of research on TDP
• (if applicable) final design mockups produced for TDP development

[ADPennington]

@elipe17 just noting for your reference that after adding missing env vars + re-staging backend, i still had to restage frontend to authenticate. im not sure if this resolves the issue described in #2839

[ADPennington]

excellent work @elipe17 thanks for all the testing support! 🚀

• added system diagram updates as a task to As OFA tech lead, I need a new permissions group for OFA data analysts  #1441
• Spike - Cloud.gov Kibana Reschedule Route Issue #2839 will capture work for troubleshooting routing issue
• Update ADR 16 with security monitoring tools in use for TDP v3 #2843 will capture updates to security tool ADR re: our use of unsafe-* policies