Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: HHS/simpler-grants-gov
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 2024.02.05-1
Choose a base ref
...
head repository: HHS/simpler-grants-gov
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: 2024.02.15-1
Choose a head ref
  • 16 commits
  • 89 files changed
  • 10 contributors

Commits on Feb 6, 2024

  1. [Issue 887] Implement WAF (#1170)

    ## Summary
    Fixes #887 
    
    ### Time to review: __x mins__
    
    ## Changes proposed
    * Add WAF and rules
    * Add logging groups for cloudwatch
    
    ## Context for reviewers
    > Implement WAF for the Simpler assets in the AWS environment.
    
    ## Additional information
    * This is currently in the console
    aplybeah authored Feb 6, 2024
    Copy the full SHA
    0ab2f7b View commit details

Commits on Feb 7, 2024

  1. [Issue 975] Google Analytics nextjs build urls (#1176)

    SammySteiner authored Feb 7, 2024
    Copy the full SHA
    0417687 View commit details
  2. [Issue 1079] Add Brett Rosenblatt to MAINTAINERS.md (#1196)

    Add Brett Rosenblatt
    
    ## Summary
    Fixes #1079 
    
    ### Time to review: __1 mins__
    
    ---------
    
    Co-authored-by: Aaron Couch <[email protected]>
    bretthrosenblatt and acouch authored Feb 7, 2024
    Copy the full SHA
    a7e8c52 View commit details

Commits on Feb 9, 2024

  1. [Issue 1135] Setup lookup value logic within the API (#1136)

    ## Summary
    Fixes #1135
    
    ### Time to review: __15 mins__
    
    ## Changes proposed
    Added the ability to configure lookup values across the API and database
    as Python enums - this includes:
    * Various utility container classes for connecting the components of the
    approach
    * A global `LookupRegistry` where we validate and store the
    configurations
    * A SQLAlchemy column type which handles converting the DB lookup IDs
    into enums automatically
    * Updated the DB migrations to automatically update the lookup tables
    * Automatically handle converting enums to integers in the DB and back
    using SQLAlchemy type decorators.
    
    ## Context for reviewers
    This implementation is largely borrowed from a prior project that used
    the template repo as well, and worked well there.
    
    While this is quite a bit, it's a lot of boilerplatey-setup, testing,
    and validation to make usage of it pretty foolproof (ie. hard to setup
    wrong). Actual usage of it should be pretty simple, just create/update
    your enum, update the config, and attach it to a table. Adding a new
    value just requires a 2-line change and the API, DB table, and all
    validation "just works". See the new `.md` file added for examples.
    
    I'm not 100% on the exact file naming/structure, but there's a lot of
    risk for circular dependencies with how it is currently setup, and this
    is what I could get to work, happy to adjust if you've got any
    suggestions.
    
    A few pre-emptive answers to questions:
    * Why use foreign keys in the DB instead of enums or check constraints?
    * Alembic struggles to detect changes to allowed values, and even a
    custom approach would be very difficult. Modifying an existing
    enum/check constraints is either complex or requires dropping and
    remaking the constraint. Also, this approach lets the ID in the DB be
    separate from the value which is nice for renaming them.
    * Why make a separate `LookupConfig` class instead of putting the value
    in the Enum/why not make a class that does both?
    * I looked into this before. The very short version is that adding more
    info to an existing Python enum is difficult, you can make the values
    something like a tuple, but it causes a lot of its convenience to break
    (can't do str to enum). Also looked into building my own equivalent to
    Python enums and that gets into complex metaclass logic, which even if
    you make a proper implementation, python typing libraries like MyPy
    struggle to interpret it like a Python enum (which under the hood is
    really really strange as each enum member is also the class itself?).
    
    ## Additional information
    Running migrations locally creates the `lk_opportunity_category` table
    and populates it with the values we've configured:
    <img width="713" alt="Screenshot 2024-02-02 at 3 16 50 PM"
    src="https://github.com/HHS/simpler-grants-gov/assets/46358556/019593c2-32be-4632-90a7-3368f11776dd">
    
    The `opportunity` table has has a foreign key column with this field:
    <img width="156" alt="Screenshot 2024-02-02 at 3 17 54 PM"
    src="https://github.com/HHS/simpler-grants-gov/assets/46358556/17d7f799-6a35-4543-8c7c-69f108187f04">
    
    Note that the factory we use to generate data locally just knows about
    the enum, the `LookupColumn` that we have configured automatically
    handles converting that to the integer without us needing to be aware of
    it.
    
    ---------
    
    Co-authored-by: nava-platform-bot <[email protected]>
    chouinar and nava-platform-bot authored Feb 9, 2024
    Copy the full SHA
    5f468ef View commit details
  2. [Issue 1166] Create tables for expanded opportunity data model (#1187)

    ## Summary
    Fixes #1166
    
    This requires #1136 to be
    merged first as it builds ontop of that work
    
    ### Time to review: __5 mins__
    
    ## Changes proposed
    Created the opportunity tables for the expanded data model
    
    https://app.gitbook.com/o/cFcvhi6d0nlLyH2VzVgn/s/v1V0jIH7mb7Yb3jlNrgk/engineering/learnings/opportunity-endpoint-data-model#overview
    
    Setup factories to generate data in a reasonable way for these tables
    
    ## Context for reviewers
    This sets up all of the tables for our expanded opportunity data model -
    tables we'll copy to for DMS that will later be transformed to these
    tables will be a separate PR.
    
    The factories try to make somewhat realistic data with some variability.
    
    
    Note that I did adjust a few things from the tech spec, which I just
    haven't changed in that document yet (can't merge the changes myself)
    - The `other_value` fields in the link tables actually make sense to
    stay in the summary table, I found examples of them working different
    than I had assumed, so it makes more sense to keep them in a general
    place.
    - The `opportunity_agency` table idea I've dropped - if we want to
    adjust how agency data is stored, will deal with it later when we deal
    with the rest of the agency data
    - Didn't adjust the primary keys - just keeping everything on
    `opportunity_id` - already was going to keep several that way - don't
    see a reason to have several different primary/foreign key setups.
    - Miscellaneous small renames to more closely match the design of the
    front-end
    
    ## Additional information
    Running `make db-seed-local` after the migrations shows the data in
    various tables in what I would expect:
    ![Screenshot 2024-02-07 at 12 55 48
    PM](https://github.com/HHS/simpler-grants-gov/assets/46358556/1432481d-0563-45d2-9660-66721c16c4ba)
    ![Screenshot 2024-02-07 at 12 56 14
    PM](https://github.com/HHS/simpler-grants-gov/assets/46358556/5d9a63fb-4339-42cc-b445-f204f9a7842c)
    ![Screenshot 2024-02-07 at 12 56 21
    PM](https://github.com/HHS/simpler-grants-gov/assets/46358556/d61999ed-f2c9-4bbe-9c09-8337fdf38914)
    ![Screenshot 2024-02-07 at 12 56 26
    PM](https://github.com/HHS/simpler-grants-gov/assets/46358556/478002bb-6c41-4429-ae6b-d088e3f8297f)
    ![Screenshot 2024-02-07 at 12 56 44
    PM](https://github.com/HHS/simpler-grants-gov/assets/46358556/2211c4cb-6f99-4655-b814-09db66b4aeb9)
    
    ---------
    
    Co-authored-by: nava-platform-bot <[email protected]>
    chouinar and nava-platform-bot authored Feb 9, 2024
    Copy the full SHA
    196ad2f View commit details

Commits on Feb 12, 2024

  1. [Issue 1216] Create the awsdms_apply_exceptions table via Alembic (#1218

    )
    
    ## Summary
    Fixes #1216
    
    ### Time to review: __3 mins__
    
    ## Changes proposed
    Created the `awsdms_apply_exceptions` table via Alembic
    
    ## Context for reviewers
    AMS DMS needs this table to run, but isn't creating it in the right
    spot, as a simple workaround, we'll create it ourselves:
    https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TaskSettings.ControlTable.html
    
    ## Additional information
    Running locally the table gets created properly:
    ![Screenshot 2024-02-12 at 4 11 42
    PM](https://github.com/HHS/simpler-grants-gov/assets/46358556/6552d070-951d-4a97-b665-575be1de48e5)
    chouinar authored Feb 12, 2024
    Copy the full SHA
    4c40197 View commit details

Commits on Feb 13, 2024

  1. Copy the full SHA
    303343f View commit details
  2. [Issue 1188] Add Brandon Tabaska to MAINTAINERS.md (#1212)

    ## Summary
    Fixes #1188 
    
    ### Time to review: __1 mins__
    btabaska authored Feb 13, 2024
    Copy the full SHA
    9279b21 View commit details
  3. [Issue 966] Configure access log S3 buckets to allow SSL requests only (

    #1219)
    
    ## Summary
    Part of #966
    
    ### Time to review: __5 mins__
    
    ## Changes proposed
    - Modify the bucket policy for the `*access-logs*` buckets to allow SSL requests only.
    
    ## Context for reviewers
    This policy is required by Security Hub control S3.5. See https://repost.aws/knowledge-center/s3-bucket-policy-for-config-rule
    
    ## Additional information
    Tested by deploying to dev using `make infra-update-app-service APP_NAME=api ENVIRONMENT=dev`.
    
    Before deploy:
    ![Screenshot 2024-02-12 at 19-20-47
    api-dev-access-logs20231023213552646900000003 - S3 bucket S3
    Global](https://github.com/HHS/simpler-grants-gov/assets/3811269/09cf21ac-c034-4da3-b41d-9726c8ba07bf)
    
    After deploy:
    ![Screenshot 2024-02-12 at 19-21-24
    api-dev-access-logs20231023213552646900000003 - S3 bucket S3
    Global](https://github.com/HHS/simpler-grants-gov/assets/3811269/9b1d5f76-8772-48ee-970d-4cf91e99d4ba)
    
    Confirmed that the control switched from FAILED to PASSED on Security
    Hub:
    ![Screenshot 2024-02-12 at 19-30-46 S3 5 Controls Security Hub
    us-east-1](https://github.com/HHS/simpler-grants-gov/assets/3811269/cacd50d2-f303-49bc-8f34-1a44885f6220)
    
    Reviewers: @coilysiren
    jamesbursa authored Feb 13, 2024
    Copy the full SHA
    f488e67 View commit details
  4. Ignore local vscode settings (#1220)

    Add .vscode* to .gitignore to match setup instructions in frontend docs
    rylew1 authored Feb 13, 2024
    Copy the full SHA
    4a08ac2 View commit details

Commits on Feb 14, 2024

  1. [Issue 1240] Create a basic ECS script which will become the copy pro…

    …cess for Oracle data (#1243)
    
    ## Summary
    Fixes #1240
    
    ### Time to review: __5 mins__
    
    ## Changes proposed
    Setup a quick script that connects to the database
    
    ## Context for reviewers
    Let me know if you think there would be a better name for any of these
    files, quickly named a lot of this.
    
    While it may seem odd, we actually can run tasks via Flask (this doesn't
    actually run routes - just re-using app setup). Running this way
    automatically handles setting up DB sessions, logging, and a few other
    utilities that we can re-use easily in a backend ECS task script. This
    shows how simple it is to set something new up (minus the infra).
    
    The actual implementation will come in a follow-up script, this is just
    to unblock any infra work and let them have something to try to run.
    
    ## Additional information
    You can run the task with `make copy-oracle-data` - for now this works
    locally and produces the following output:
    ![Screenshot 2024-02-14 at 3 46 10
    PM](https://github.com/HHS/simpler-grants-gov/assets/46358556/e3271fc6-b10d-4f4e-ae7c-d59ffc3cb46c)
    chouinar authored Feb 14, 2024
    Copy the full SHA
    ed4cc06 View commit details

Commits on Feb 15, 2024

  1. Update React (#1208)

    renovate[bot] authored Feb 15, 2024
    Copy the full SHA
    8ea806d View commit details
  2. [Issue 1165] Setup the v0.1 opportunity endpoints (#1213)

    ## Summary
    Fixes #1165
    
    ### Time to review: __15 mins__
    
    ## Changes proposed
    Added a new set of `v0.1` opportunity endpoints which are connected to
    the DB tables, but don't include filtering yet
    
    Added schemas for the expanded opportunity model
    
    A slight adjustment in the structure of pagination in the request
    
    Quite a bit of restructuring/renaming to have versioning be clearer
    in-code
    
    Add an environment variable for enabling the v0.1 endpoint - only
    enabled locally at the moment
    
    ## Context for reviewers
    There are follow-up tickets to setup the endpoint including adding
    filters and mock data. This PR was already pretty big so aimed to just
    get things connected and restructured and will later build something a
    bit more meaningful.
    
    The main thing I did with the renaming was to move the routes+schemas
    into folders for their specific versions. This way we can avoid naming
    most of the variables `opportunity_blueprint_v0_1` or similar. However,
    the class name of Marshmallow schemas is registered globally in their
    internal logic so we can't have two separate `OpportunitySchema`
    classes. I'm hoping we won't need another significant version of the
    opportunity schemas, so preferred naming the existing one `v0` and
    leaving the version out of the new one.
    
    ## Additional information
    <img width="1419" alt="Screenshot 2024-02-12 at 1 40 00 PM"
    src="https://github.com/HHS/simpler-grants-gov/assets/46358556/248876d8-9640-4187-8771-566bb63d2298">
    
    The endpoints work locally with the database, returning data as
    expected. Click the sections below to see the responses.
    
    <details>
    <summary>Click me to see a GET opportunity response</summary>
    
    ```json
    {
      "data": {
        "agency": "US-ABC",
        "applicant_types": [
          "federally_recognized_native_american_tribal_governments",
          "other_native_american_tribal_organizations"
        ],
        "category": "continuation",
        "category_explanation": null,
        "created_at": "2024-02-12T18:40:35.832388+00:00",
        "funding_categories": [
          "education",
          "employment_labor_and_training"
        ],
        "funding_instruments": [
          "other"
        ],
        "modified_comments": null,
        "opportunity_assistance_listings": [
          {
            "assistance_listing_number": "16.870",
            "program_title": "Hill Group"
          },
          {
            "assistance_listing_number": "59.831",
            "program_title": "Wall and Sons"
          },
          {
            "assistance_listing_number": "10.403",
            "program_title": "Dixon, Salas and Gonzalez"
          }
        ],
        "opportunity_id": 4,
        "opportunity_number": "ABC-4-XYZ-001",
        "opportunity_title": "Research into Field seismologist industry",
        "revision_number": 0,
        "summary": {
          "additional_info_url": "grants.gov",
          "additional_info_url_description": "Click me for additional info",
          "agency_code": "US-ABC",
          "agency_contact_description": "For more information contact us at - Foreign reach minute evening environment option form. Serve pick land management lot international. Field news finish fly result.",
          "agency_email_address": "[email protected]",
          "agency_email_address_description": "Contact US Alphabetical Basic Corp via email",
          "agency_name": "US Alphabetical Basic Corp",
          "agency_phone_number": "123-456-0003",
          "applicant_eligibility_description": "Property today share serious inside hotel argue. Look final material product management. Particular food compare recognize factor reduce option your. Box garden then plant offer this man. Back wife open pull. House firm hear true.",
          "archive_date": "2024-01-27",
          "award_ceiling": 8000000,
          "award_floor": 2666667,
          "close_date": "2024-01-24",
          "close_date_description": "Debate draw show science firm station into.",
          "estimated_total_program_funding": 8000000,
          "expected_number_of_awards": 3,
          "funding_category_description": null,
          "is_cost_sharing": false,
          "opportunity_status": "archived",
          "post_date": "2024-02-03",
          "summary_description": "Example summary - Week adult hair factor. Court itself end six music especially."
        },
        "updated_at": "2024-02-12T18:40:35.832388+00:00"
      },
      "message": "Success",
      "pagination_info": null,
      "status_code": 200,
      "warnings": []
    }
    ```
    
    </details>
    
    <details>
    <summary>Click me to see a POST opportunities (search)
    response</summary>
    
    ```json
    {
      "data": [
        {
          "agency": "US-ABC",
          "applicant_types": [
            "unrestricted",
            "other"
          ],
          "category": "discretionary",
          "category_explanation": null,
          "created_at": "2024-02-12T18:40:36.086758+00:00",
          "funding_categories": [
            "income_security_and_social_services",
            "law_justice_and_legal_services",
            "natural_resources"
          ],
          "funding_instruments": [
            "grant",
            "cooperative_agreement",
            "other"
          ],
          "modified_comments": null,
          "opportunity_assistance_listings": [
            {
              "assistance_listing_number": "01.878",
              "program_title": "Hill, Chambers and Brown"
            },
            {
              "assistance_listing_number": "43.779",
              "program_title": "Mcneil Ltd"
            }
          ],
          "opportunity_id": 10,
          "opportunity_number": "ABC-10-XYZ-001",
          "opportunity_title": "Research into Engineer, automotive industry",
          "revision_number": 0,
          "summary": {
            "additional_info_url": "grants.gov",
            "additional_info_url_description": "Click me for additional info",
            "agency_code": "US-ABC",
            "agency_contact_description": "For more information contact us at - Ball support player real billion.",
            "agency_email_address": "[email protected]",
            "agency_email_address_description": "Contact US Alphabetical Basic Corp via email",
            "agency_name": "US Alphabetical Basic Corp",
            "agency_phone_number": "123-456-0009",
            "applicant_eligibility_description": null,
            "archive_date": "2024-02-29",
            "award_ceiling": 4175000,
            "award_floor": 278333,
            "close_date": "2024-02-28",
            "close_date_description": "Maintain door weight poor adult.",
            "estimated_total_program_funding": 4175000,
            "expected_number_of_awards": 15,
            "funding_category_description": "New successful different enter hundred might building week.",
            "is_cost_sharing": true,
            "opportunity_status": "forecasted",
            "post_date": "2024-02-10",
            "summary_description": "Example summary - Environment home herself reflect summer act life at. Admit suffer doctor pick."
          },
          "updated_at": "2024-02-12T18:40:36.086758+00:00"
        },
        {
          "agency": "US-XYZ",
          "applicant_types": [
            "county_governments",
            "state_governments"
          ],
          "category": "other",
          "category_explanation": "Category as chosen by order #120",
          "created_at": "2024-02-12T18:40:36.150937+00:00",
          "funding_categories": [
            "opportunity_zone_benefits",
            "regional_development",
            "science_technology_and_other_research_and_development"
          ],
          "funding_instruments": [
            "cooperative_agreement",
            "other",
            "procurement_contract"
          ],
          "modified_comments": null,
          "opportunity_assistance_listings": [
            {
              "assistance_listing_number": "89.273",
              "program_title": "Foster, Brown and Ray"
            },
            {
              "assistance_listing_number": "85.071",
              "program_title": "Bell and Sons"
            }
          ],
          "opportunity_id": 11,
          "opportunity_number": "ABC-11-XYZ-001",
          "opportunity_title": "Research into Occupational hygienist industry",
          "revision_number": 0,
          "summary": {
            "additional_info_url": "google.com",
            "additional_info_url_description": "Click me for additional info",
            "agency_code": "US-XYZ",
            "agency_contact_description": "For more information contact us at - Available assume expert although ten recently book. Security that would on recent soon.",
            "agency_email_address": "[email protected]",
            "agency_email_address_description": "Contact US Xylophone Yak Zoo via email",
            "agency_name": "US Xylophone Yak Zoo",
            "agency_phone_number": "123-456-0010",
            "applicant_eligibility_description": "Onto might cut simply collection if. Believe likely task question somebody. Around carry interest bed probably full article. Must game analysis type not weight give. Property care miss any this.",
            "archive_date": "2024-01-25",
            "award_ceiling": 7530000,
            "award_floor": 376500,
            "close_date": "2024-01-23",
            "close_date_description": "Three few draw boy thousand seven.",
            "estimated_total_program_funding": 7530000,
            "expected_number_of_awards": 20,
            "funding_category_description": "Middle wear gun sit themselves.",
            "is_cost_sharing": true,
            "opportunity_status": "archived",
            "post_date": "2024-01-30",
            "summary_description": "Example summary - Worry why science when. Door amount majority situation. Item reduce thus."
          },
          "updated_at": "2024-02-12T18:40:36.150937+00:00"
        },
        {
          "agency": "US-123",
          "applicant_types": [
            "city_or_township_governments"
          ],
          "category": "earmark",
          "category_explanation": null,
          "created_at": "2024-02-12T18:40:36.225715+00:00",
          "funding_categories": [
            "transportation",
            "affordable_care_act",
            "other"
          ],
          "funding_instruments": [
            "grant"
          ],
          "modified_comments": null,
          "opportunity_assistance_listings": [
            {
              "assistance_listing_number": "16.711",
              "program_title": "Gonzalez PLC"
            },
            {
              "assistance_listing_number": "38.607",
              "program_title": "Roberson, Brown and Henderson"
            },
            {
              "assistance_listing_number": "69.310",
              "program_title": "House, Blackwell and Zuniga"
            }
          ],
          "opportunity_id": 12,
          "opportunity_number": "ABC-12-XYZ-001",
          "opportunity_title": "Research into Commercial/residential surveyor industry",
          "revision_number": 0,
          "summary": {
            "additional_info_url": "grants.gov",
            "additional_info_url_description": "Click me for additional info",
            "agency_code": "US-123",
            "agency_contact_description": "For more information contact us at - Choice record answer mention when east somebody. Person significant pick explain often case. Weight rate these cultural part age admit.",
            "agency_email_address": "[email protected]",
            "agency_email_address_description": "Contact US Number Department via email",
            "agency_name": "US Number Department",
            "agency_phone_number": "123-456-0011",
            "applicant_eligibility_description": "Bill skin indeed item mind win accept. Mrs professional late company sound. Rich source interesting success.",
            "archive_date": "2024-02-28",
            "award_ceiling": 2390000,
            "award_floor": 199167,
            "close_date": "2024-03-02",
            "close_date_description": "Explain prepare along company.",
            "estimated_total_program_funding": 2390000,
            "expected_number_of_awards": 12,
            "funding_category_description": "Such subject set forget him poor arm.",
            "is_cost_sharing": true,
            "opportunity_status": "forecasted",
            "post_date": "2024-01-31",
            "summary_description": "Example summary - Say night charge onto agree study. Thank agency boy know."
          },
          "updated_at": "2024-02-12T18:40:36.225715+00:00"
        }
      ],
      "message": "Success",
      "pagination_info": {
        "order_by": "opportunity_id",
        "page_offset": 4,
        "page_size": 3,
        "sort_direction": "ascending",
        "total_pages": 9,
        "total_records": 25
      },
      "status_code": 200,
      "warnings": []
    }
    ```
    
    </details>
    
    ---------
    
    Co-authored-by: nava-platform-bot <[email protected]>
    chouinar and nava-platform-bot authored Feb 15, 2024
    Copy the full SHA
    170e7db View commit details
  3. Copy the full SHA
    5e0beb9 View commit details
  4. Copy the full SHA
    24b8295 View commit details
  5. Copy the full SHA
    5256754 View commit details
Loading