Outlier AWS CDK Infrastructure (Python)

Overview

This repository contains Outlier's AWS CDK project. It defines infrastructure as code for the organization's Nightly AWS account using AWS CDK in Python. The project leverages AWS CDK, Projen, and GitHub Actions for streamlined and efficient deployments to Outlier's AWS environments.

---

Features

• Streamlined Setup: Quickly configure the project through a single configuration file (.projenrc.py).
• Multi-Account Support: Provides flexibility for managing multiple AWS accounts across various environments.
• Automated Deployment Pipelines: Pre-configured GitHub Actions workflows automate deployment processes.
• Organized Project Structure: Logical and intuitive structure for managing constructs and stacks.
• Secure Deployments: Utilizes OpenID Connect for secure, credential-less GitHub Actions authentication with AWS.
• Dependency Management: Handles dependencies and virtual environments using Poetry.
• Fast Linting and Formatting: Includes Ruff for efficient linting and formatting.
• Enhanced PR Process: Built-in pull request templates streamline code reviews.

---

Which Outlier AWS Resources are ✅ managed by this project?

• Our core application stack.
  • ✅ ECR (App Images)
  • ✅ ALB (App Load Balancer)
  • ✅ ECS (App Containers)
  • ✅ RDS (App Database)
  • ✅ CodePipeline (App CI/CD)
  • ✅ S3 Buckets for Application (App Blob Storage)

Which Outlier AWS Resources are NOT ❌ managed by this project?

• Any non-core application resources.
  • ❌ VPC and other high-level networking resources
    • Why? Savvas IFT manages our high-level networking resources themselves, through Iaac (Terraform/CDK). We do not want to have 2 separate IaaC projects trying to manage the same resources.
    • Because of this, we are not and SHOULD NOT be managing any AWS Resources that have terraform_managed = True as a tag.
    • We do, however, dynamically import and reference these values in this project.
  • ❌ Task Definitions
    • Why? These live inside our application repositories and are dynamically generated and used by our AWS CodePipeline. see outlier-api/taskdef_nightly.json
  • ❌ Secrets Manager
    • Why? It is not good practice to manage Secrets Manager resources in this code.
    • We do, however, dynamically fetch/import and reference these values in this project as needed.
  • ❌ ACM Certificates
    • Why? Certificates often have their own lifecycle outside of the core application resources, sometimes with other Savvas parties needing to make changes to them. Because of this, I chose to leave their management in the AWS console.
    • We do, however, dynamically import and reference these values in this project.
  • ❌ Redshift (Data Warehouse)
  • ❌ Firehose, DMS, DataSync and other non-application-stack services.

---

Setup Instructions

Steps to Configure and Deploy

1. Clone the Repository: Clone this repository to a local environment.
2. Configure GitHub Access: Add a Personal Access Token in the repository settings on GitHub following these instructions.
3. Install Required Tools: Install AWS CDK and Projen globally:

   npm install -g aws-cdk projen

4. Install Dependencies: Install project dependencies with Poetry:

   poetry install

5. Configure Project Settings: Modify the AWS region and account IDs in the .projenrc.py file:

   aws_region = os.getenv("AWS_REGION", "us-east-1")
   target_accounts = {
       "dev": "987654321012",
       "test": "123456789012",
       "staging": None,
       "production": None,
   }

6. Generate Workflow Files: Run Projen to generate GitHub Actions workflow files:

   projen

7. Authenticate AWS CLI: Log in to the appropriate AWS account using the AWS CLI. Follow this guide if necessary.
8. Bootstrap CDK Environment: Deploy the CDK toolkit stack if not already set up:

   cdk bootstrap

9. Deploy GitHub OIDC Stack: Enable GitHub Actions to deploy resources by executing:

   projen dev:deploy

10. Commit and Push Changes: Push changes to the main branch to trigger the deployment pipeline.

---

Project Structure

The project is organized into logical units to facilitate maintainability and scalability:

.
├── cdk.json
├── poetry.lock
├── pyproject.toml
├── README.md
├── src
│  ├── __init__.py
│  ├── app.py
│  ├── assets
│  │  ├── ecs
│  │  │  └── hello-world
│  │  │     └── Dockerfile
│  │  └── lambda
│  │     └── hello-world
│  │        └── lambda_function.py
│  ├── bin
│  │  ├── cicd_helper.py
│  │  ├── env_helper.py
│  │  └── git_helper.py
│  ├── custom_constructs
│  │  ├── __init__.py
│  │  ├── base_construct.py
│  │  ├── network_construct.py
│  │  └── README.md
│  └── stacks
│     ├── __init__.py
│     ├── base_stack.py
│     ├── github_oidc_stack.py
│     └── README.md
└── tests
   ├── __init__.py
   └── test_example.py

Section Details

• src/assets: Contains application code for Lambda functions and ECS services.
• src/bin: Includes utility scripts for environment setup and CI/CD integration.
• src/custom_constructs: Houses reusable constructs for infrastructure components.
• src/stacks: Defines AWS stacks for deploying collections of resources.
• tests: Contains unit and integration tests.

This structure ensures maintainability, scalability, and efficient collaboration across Outlier's infrastructure projects.