-
Notifications
You must be signed in to change notification settings - Fork 19
Comparing changes
Open a pull request
base repository: HHS/simpler-grants-gov
base: 2024.02.05-1
head repository: HHS/simpler-grants-gov
compare: 2024.02.15-1
- 16 commits
- 89 files changed
- 10 contributors
Commits on Feb 6, 2024
-
[Issue 887] Implement WAF (#1170)
## Summary Fixes #887 ### Time to review: __x mins__ ## Changes proposed * Add WAF and rules * Add logging groups for cloudwatch ## Context for reviewers > Implement WAF for the Simpler assets in the AWS environment. ## Additional information * This is currently in the console
Configuration menu - View commit details
-
Copy full SHA for 0ab2f7b - Browse repository at this point
Copy the full SHA 0ab2f7bView commit details
Commits on Feb 7, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 0417687 - Browse repository at this point
Copy the full SHA 0417687View commit details -
[Issue 1079] Add Brett Rosenblatt to MAINTAINERS.md (#1196)
Add Brett Rosenblatt ## Summary Fixes #1079 ### Time to review: __1 mins__ --------- Co-authored-by: Aaron Couch <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for a7e8c52 - Browse repository at this point
Copy the full SHA a7e8c52View commit details
Commits on Feb 9, 2024
-
[Issue 1135] Setup lookup value logic within the API (#1136)
## Summary Fixes #1135 ### Time to review: __15 mins__ ## Changes proposed Added the ability to configure lookup values across the API and database as Python enums - this includes: * Various utility container classes for connecting the components of the approach * A global `LookupRegistry` where we validate and store the configurations * A SQLAlchemy column type which handles converting the DB lookup IDs into enums automatically * Updated the DB migrations to automatically update the lookup tables * Automatically handle converting enums to integers in the DB and back using SQLAlchemy type decorators. ## Context for reviewers This implementation is largely borrowed from a prior project that used the template repo as well, and worked well there. While this is quite a bit, it's a lot of boilerplatey-setup, testing, and validation to make usage of it pretty foolproof (ie. hard to setup wrong). Actual usage of it should be pretty simple, just create/update your enum, update the config, and attach it to a table. Adding a new value just requires a 2-line change and the API, DB table, and all validation "just works". See the new `.md` file added for examples. I'm not 100% on the exact file naming/structure, but there's a lot of risk for circular dependencies with how it is currently setup, and this is what I could get to work, happy to adjust if you've got any suggestions. A few pre-emptive answers to questions: * Why use foreign keys in the DB instead of enums or check constraints? * Alembic struggles to detect changes to allowed values, and even a custom approach would be very difficult. Modifying an existing enum/check constraints is either complex or requires dropping and remaking the constraint. Also, this approach lets the ID in the DB be separate from the value which is nice for renaming them. * Why make a separate `LookupConfig` class instead of putting the value in the Enum/why not make a class that does both? * I looked into this before. The very short version is that adding more info to an existing Python enum is difficult, you can make the values something like a tuple, but it causes a lot of its convenience to break (can't do str to enum). Also looked into building my own equivalent to Python enums and that gets into complex metaclass logic, which even if you make a proper implementation, python typing libraries like MyPy struggle to interpret it like a Python enum (which under the hood is really really strange as each enum member is also the class itself?). ## Additional information Running migrations locally creates the `lk_opportunity_category` table and populates it with the values we've configured: <img width="713" alt="Screenshot 2024-02-02 at 3 16 50 PM" src="https://github.com/HHS/simpler-grants-gov/assets/46358556/019593c2-32be-4632-90a7-3368f11776dd"> The `opportunity` table has has a foreign key column with this field: <img width="156" alt="Screenshot 2024-02-02 at 3 17 54 PM" src="https://github.com/HHS/simpler-grants-gov/assets/46358556/17d7f799-6a35-4543-8c7c-69f108187f04"> Note that the factory we use to generate data locally just knows about the enum, the `LookupColumn` that we have configured automatically handles converting that to the integer without us needing to be aware of it. --------- Co-authored-by: nava-platform-bot <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 5f468ef - Browse repository at this point
Copy the full SHA 5f468efView commit details -
[Issue 1166] Create tables for expanded opportunity data model (#1187)
## Summary Fixes #1166 This requires #1136 to be merged first as it builds ontop of that work ### Time to review: __5 mins__ ## Changes proposed Created the opportunity tables for the expanded data model https://app.gitbook.com/o/cFcvhi6d0nlLyH2VzVgn/s/v1V0jIH7mb7Yb3jlNrgk/engineering/learnings/opportunity-endpoint-data-model#overview Setup factories to generate data in a reasonable way for these tables ## Context for reviewers This sets up all of the tables for our expanded opportunity data model - tables we'll copy to for DMS that will later be transformed to these tables will be a separate PR. The factories try to make somewhat realistic data with some variability. Note that I did adjust a few things from the tech spec, which I just haven't changed in that document yet (can't merge the changes myself) - The `other_value` fields in the link tables actually make sense to stay in the summary table, I found examples of them working different than I had assumed, so it makes more sense to keep them in a general place. - The `opportunity_agency` table idea I've dropped - if we want to adjust how agency data is stored, will deal with it later when we deal with the rest of the agency data - Didn't adjust the primary keys - just keeping everything on `opportunity_id` - already was going to keep several that way - don't see a reason to have several different primary/foreign key setups. - Miscellaneous small renames to more closely match the design of the front-end ## Additional information Running `make db-seed-local` after the migrations shows the data in various tables in what I would expect:      --------- Co-authored-by: nava-platform-bot <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 196ad2f - Browse repository at this point
Copy the full SHA 196ad2fView commit details
Commits on Feb 12, 2024
-
[Issue 1216] Create the awsdms_apply_exceptions table via Alembic (#1218
) ## Summary Fixes #1216 ### Time to review: __3 mins__ ## Changes proposed Created the `awsdms_apply_exceptions` table via Alembic ## Context for reviewers AMS DMS needs this table to run, but isn't creating it in the right spot, as a simple workaround, we'll create it ourselves: https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TaskSettings.ControlTable.html ## Additional information Running locally the table gets created properly: 
Configuration menu - View commit details
-
Copy full SHA for 4c40197 - Browse repository at this point
Copy the full SHA 4c40197View commit details
Commits on Feb 13, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 303343f - Browse repository at this point
Copy the full SHA 303343fView commit details -
[Issue 1188] Add Brandon Tabaska to MAINTAINERS.md (#1212)
## Summary Fixes #1188 ### Time to review: __1 mins__
Configuration menu - View commit details
-
Copy full SHA for 9279b21 - Browse repository at this point
Copy the full SHA 9279b21View commit details -
[Issue 966] Configure access log S3 buckets to allow SSL requests only (
#1219) ## Summary Part of #966 ### Time to review: __5 mins__ ## Changes proposed - Modify the bucket policy for the `*access-logs*` buckets to allow SSL requests only. ## Context for reviewers This policy is required by Security Hub control S3.5. See https://repost.aws/knowledge-center/s3-bucket-policy-for-config-rule ## Additional information Tested by deploying to dev using `make infra-update-app-service APP_NAME=api ENVIRONMENT=dev`. Before deploy:  After deploy:  Confirmed that the control switched from FAILED to PASSED on Security Hub:  Reviewers: @coilysiren
Configuration menu - View commit details
-
Copy full SHA for f488e67 - Browse repository at this point
Copy the full SHA f488e67View commit details -
Ignore local vscode settings (#1220)
Add .vscode* to .gitignore to match setup instructions in frontend docs
Configuration menu - View commit details
-
Copy full SHA for 4a08ac2 - Browse repository at this point
Copy the full SHA 4a08ac2View commit details
Commits on Feb 14, 2024
-
[Issue 1240] Create a basic ECS script which will become the copy pro…
…cess for Oracle data (#1243) ## Summary Fixes #1240 ### Time to review: __5 mins__ ## Changes proposed Setup a quick script that connects to the database ## Context for reviewers Let me know if you think there would be a better name for any of these files, quickly named a lot of this. While it may seem odd, we actually can run tasks via Flask (this doesn't actually run routes - just re-using app setup). Running this way automatically handles setting up DB sessions, logging, and a few other utilities that we can re-use easily in a backend ECS task script. This shows how simple it is to set something new up (minus the infra). The actual implementation will come in a follow-up script, this is just to unblock any infra work and let them have something to try to run. ## Additional information You can run the task with `make copy-oracle-data` - for now this works locally and produces the following output: 
Configuration menu - View commit details
-
Copy full SHA for ed4cc06 - Browse repository at this point
Copy the full SHA ed4cc06View commit details
Commits on Feb 15, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 8ea806d - Browse repository at this point
Copy the full SHA 8ea806dView commit details -
[Issue 1165] Setup the v0.1 opportunity endpoints (#1213)
## Summary Fixes #1165 ### Time to review: __15 mins__ ## Changes proposed Added a new set of `v0.1` opportunity endpoints which are connected to the DB tables, but don't include filtering yet Added schemas for the expanded opportunity model A slight adjustment in the structure of pagination in the request Quite a bit of restructuring/renaming to have versioning be clearer in-code Add an environment variable for enabling the v0.1 endpoint - only enabled locally at the moment ## Context for reviewers There are follow-up tickets to setup the endpoint including adding filters and mock data. This PR was already pretty big so aimed to just get things connected and restructured and will later build something a bit more meaningful. The main thing I did with the renaming was to move the routes+schemas into folders for their specific versions. This way we can avoid naming most of the variables `opportunity_blueprint_v0_1` or similar. However, the class name of Marshmallow schemas is registered globally in their internal logic so we can't have two separate `OpportunitySchema` classes. I'm hoping we won't need another significant version of the opportunity schemas, so preferred naming the existing one `v0` and leaving the version out of the new one. ## Additional information <img width="1419" alt="Screenshot 2024-02-12 at 1 40 00 PM" src="https://github.com/HHS/simpler-grants-gov/assets/46358556/248876d8-9640-4187-8771-566bb63d2298"> The endpoints work locally with the database, returning data as expected. Click the sections below to see the responses. <details> <summary>Click me to see a GET opportunity response</summary> ```json { "data": { "agency": "US-ABC", "applicant_types": [ "federally_recognized_native_american_tribal_governments", "other_native_american_tribal_organizations" ], "category": "continuation", "category_explanation": null, "created_at": "2024-02-12T18:40:35.832388+00:00", "funding_categories": [ "education", "employment_labor_and_training" ], "funding_instruments": [ "other" ], "modified_comments": null, "opportunity_assistance_listings": [ { "assistance_listing_number": "16.870", "program_title": "Hill Group" }, { "assistance_listing_number": "59.831", "program_title": "Wall and Sons" }, { "assistance_listing_number": "10.403", "program_title": "Dixon, Salas and Gonzalez" } ], "opportunity_id": 4, "opportunity_number": "ABC-4-XYZ-001", "opportunity_title": "Research into Field seismologist industry", "revision_number": 0, "summary": { "additional_info_url": "grants.gov", "additional_info_url_description": "Click me for additional info", "agency_code": "US-ABC", "agency_contact_description": "For more information contact us at - Foreign reach minute evening environment option form. Serve pick land management lot international. Field news finish fly result.", "agency_email_address": "[email protected]", "agency_email_address_description": "Contact US Alphabetical Basic Corp via email", "agency_name": "US Alphabetical Basic Corp", "agency_phone_number": "123-456-0003", "applicant_eligibility_description": "Property today share serious inside hotel argue. Look final material product management. Particular food compare recognize factor reduce option your. Box garden then plant offer this man. Back wife open pull. House firm hear true.", "archive_date": "2024-01-27", "award_ceiling": 8000000, "award_floor": 2666667, "close_date": "2024-01-24", "close_date_description": "Debate draw show science firm station into.", "estimated_total_program_funding": 8000000, "expected_number_of_awards": 3, "funding_category_description": null, "is_cost_sharing": false, "opportunity_status": "archived", "post_date": "2024-02-03", "summary_description": "Example summary - Week adult hair factor. Court itself end six music especially." }, "updated_at": "2024-02-12T18:40:35.832388+00:00" }, "message": "Success", "pagination_info": null, "status_code": 200, "warnings": [] } ``` </details> <details> <summary>Click me to see a POST opportunities (search) response</summary> ```json { "data": [ { "agency": "US-ABC", "applicant_types": [ "unrestricted", "other" ], "category": "discretionary", "category_explanation": null, "created_at": "2024-02-12T18:40:36.086758+00:00", "funding_categories": [ "income_security_and_social_services", "law_justice_and_legal_services", "natural_resources" ], "funding_instruments": [ "grant", "cooperative_agreement", "other" ], "modified_comments": null, "opportunity_assistance_listings": [ { "assistance_listing_number": "01.878", "program_title": "Hill, Chambers and Brown" }, { "assistance_listing_number": "43.779", "program_title": "Mcneil Ltd" } ], "opportunity_id": 10, "opportunity_number": "ABC-10-XYZ-001", "opportunity_title": "Research into Engineer, automotive industry", "revision_number": 0, "summary": { "additional_info_url": "grants.gov", "additional_info_url_description": "Click me for additional info", "agency_code": "US-ABC", "agency_contact_description": "For more information contact us at - Ball support player real billion.", "agency_email_address": "[email protected]", "agency_email_address_description": "Contact US Alphabetical Basic Corp via email", "agency_name": "US Alphabetical Basic Corp", "agency_phone_number": "123-456-0009", "applicant_eligibility_description": null, "archive_date": "2024-02-29", "award_ceiling": 4175000, "award_floor": 278333, "close_date": "2024-02-28", "close_date_description": "Maintain door weight poor adult.", "estimated_total_program_funding": 4175000, "expected_number_of_awards": 15, "funding_category_description": "New successful different enter hundred might building week.", "is_cost_sharing": true, "opportunity_status": "forecasted", "post_date": "2024-02-10", "summary_description": "Example summary - Environment home herself reflect summer act life at. Admit suffer doctor pick." }, "updated_at": "2024-02-12T18:40:36.086758+00:00" }, { "agency": "US-XYZ", "applicant_types": [ "county_governments", "state_governments" ], "category": "other", "category_explanation": "Category as chosen by order #120", "created_at": "2024-02-12T18:40:36.150937+00:00", "funding_categories": [ "opportunity_zone_benefits", "regional_development", "science_technology_and_other_research_and_development" ], "funding_instruments": [ "cooperative_agreement", "other", "procurement_contract" ], "modified_comments": null, "opportunity_assistance_listings": [ { "assistance_listing_number": "89.273", "program_title": "Foster, Brown and Ray" }, { "assistance_listing_number": "85.071", "program_title": "Bell and Sons" } ], "opportunity_id": 11, "opportunity_number": "ABC-11-XYZ-001", "opportunity_title": "Research into Occupational hygienist industry", "revision_number": 0, "summary": { "additional_info_url": "google.com", "additional_info_url_description": "Click me for additional info", "agency_code": "US-XYZ", "agency_contact_description": "For more information contact us at - Available assume expert although ten recently book. Security that would on recent soon.", "agency_email_address": "[email protected]", "agency_email_address_description": "Contact US Xylophone Yak Zoo via email", "agency_name": "US Xylophone Yak Zoo", "agency_phone_number": "123-456-0010", "applicant_eligibility_description": "Onto might cut simply collection if. Believe likely task question somebody. Around carry interest bed probably full article. Must game analysis type not weight give. Property care miss any this.", "archive_date": "2024-01-25", "award_ceiling": 7530000, "award_floor": 376500, "close_date": "2024-01-23", "close_date_description": "Three few draw boy thousand seven.", "estimated_total_program_funding": 7530000, "expected_number_of_awards": 20, "funding_category_description": "Middle wear gun sit themselves.", "is_cost_sharing": true, "opportunity_status": "archived", "post_date": "2024-01-30", "summary_description": "Example summary - Worry why science when. Door amount majority situation. Item reduce thus." }, "updated_at": "2024-02-12T18:40:36.150937+00:00" }, { "agency": "US-123", "applicant_types": [ "city_or_township_governments" ], "category": "earmark", "category_explanation": null, "created_at": "2024-02-12T18:40:36.225715+00:00", "funding_categories": [ "transportation", "affordable_care_act", "other" ], "funding_instruments": [ "grant" ], "modified_comments": null, "opportunity_assistance_listings": [ { "assistance_listing_number": "16.711", "program_title": "Gonzalez PLC" }, { "assistance_listing_number": "38.607", "program_title": "Roberson, Brown and Henderson" }, { "assistance_listing_number": "69.310", "program_title": "House, Blackwell and Zuniga" } ], "opportunity_id": 12, "opportunity_number": "ABC-12-XYZ-001", "opportunity_title": "Research into Commercial/residential surveyor industry", "revision_number": 0, "summary": { "additional_info_url": "grants.gov", "additional_info_url_description": "Click me for additional info", "agency_code": "US-123", "agency_contact_description": "For more information contact us at - Choice record answer mention when east somebody. Person significant pick explain often case. Weight rate these cultural part age admit.", "agency_email_address": "[email protected]", "agency_email_address_description": "Contact US Number Department via email", "agency_name": "US Number Department", "agency_phone_number": "123-456-0011", "applicant_eligibility_description": "Bill skin indeed item mind win accept. Mrs professional late company sound. Rich source interesting success.", "archive_date": "2024-02-28", "award_ceiling": 2390000, "award_floor": 199167, "close_date": "2024-03-02", "close_date_description": "Explain prepare along company.", "estimated_total_program_funding": 2390000, "expected_number_of_awards": 12, "funding_category_description": "Such subject set forget him poor arm.", "is_cost_sharing": true, "opportunity_status": "forecasted", "post_date": "2024-01-31", "summary_description": "Example summary - Say night charge onto agree study. Thank agency boy know." }, "updated_at": "2024-02-12T18:40:36.225715+00:00" } ], "message": "Success", "pagination_info": { "order_by": "opportunity_id", "page_offset": 4, "page_size": 3, "sort_direction": "ascending", "total_pages": 9, "total_records": 25 }, "status_code": 200, "warnings": [] } ``` </details> --------- Co-authored-by: nava-platform-bot <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 170e7db - Browse repository at this point
Copy the full SHA 170e7dbView commit details -
Configuration menu - View commit details
-
Copy full SHA for 5e0beb9 - Browse repository at this point
Copy the full SHA 5e0beb9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 24b8295 - Browse repository at this point
Copy the full SHA 24b8295View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5256754 - Browse repository at this point
Copy the full SHA 5256754View commit details
This comparison is taking too long to generate.
Unfortunately it looks like we can’t render this comparison for you right now. It might be too big, or there might be something weird with your repository.
You can try running this command locally to see the comparison on your machine:
git diff 2024.02.05-1...2024.02.15-1