Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge product decision records into architecture decision records #2728

Merged
merged 1 commit into from
Nov 6, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# 1. Record product decisions
# 5. Record product decisions

Introduced: 20220330
Introduced: 2022-03-30

## Status
Accepted
Expand All @@ -15,3 +15,6 @@ The context/decision/consequences framework should work for us; we will alter th

## Consequences
See Michael Nygard's article, linked above.

## Notes
This was originally product decision record 0001; renamed/renumbered as part of merging ADRs and PDRs.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# 2. Capture only Universal Entity Identifier (UEI)
# 6. Capture only Universal Entity Identifier (UEI)

Introduced: 20220411
Introduced: 2022-04-11

## Status

Expand Down Expand Up @@ -39,4 +39,7 @@ Historical connection to DUNS numbers is out of scope for the MVP. More research

The community has never submitted this identifer to the SF-SAC before. This will change the user's experience, but not their reporting burden, because *they should already have a UEI*. That said, we know that auditors are currently preparing the SF-SAC based on *last year's* form, and therefore they are not preparing grantees for this new number, or the UX changes the UEI represent.

So, there are consequences, but our steering committee (and internal discussion on the team) confirms that this is the correct path for the FAC MVP.
So, there are consequences, but our steering committee (and internal discussion on the team) confirms that this is the correct path for the FAC MVP.

## Notes
This was originally product decision record 0002; renamed/renumbered as part of merging ADRs and PDRs.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# 5. Use Github Actions for CI
# 7. Use Github Actions for CI

Date: 2022-04-12

Expand Down Expand Up @@ -31,3 +31,6 @@ Finding no issues with either option and opting to stick with Github-native work
* We'll block merging to main on passing tests and a successful build.
* We'll automatically deploy the latest passing build to cloud.gov
* We'll continue to build out CI as we add linting, security scanning, accessibility scanning, etc.

## Note
Was previously ADR 0005; renamed/renumbered when PDRs and ADRs were merged.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# 3. Make `gsa-tts/fac` repository public
# 8. Make `gsa-tts/fac` repository public

Introduced: 20220420
Introduced: 2022-04-20

## Status

Expand All @@ -22,3 +22,5 @@ Switch `GSA-TTS/FAC` from private to public, and leave `GSA-TTS/FAC-Frontend` pu

We may need, for one reason or another, make one or both repositories private at some point in the future. At that point, we will discuss and write another PDR documenting our decision.

## Note
Was previously PDR 0005; renamed/renumbered when PDRs and ADRs were merged.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# 6. Use the Static Site Generator Eleventy for the FAC Frontend
# 9. Use the Static Site Generator Eleventy for the FAC Frontend

Date: 2022-04-26

Expand All @@ -25,3 +25,6 @@ The TTS Engineering Practices Guide to [Choosing a Web App Architecture](https:/

* Client-side interactions should be simple.
* The server will be responsible for managing application state.

## Note
Was previously ADR 0006; renamed/renumbered when PDRs and ADRs were merged.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# 7. Use the SAM.gov API for UEI validation
# 10. Use the SAM.gov API for UEI validation

Date: 2022-05-10

Expand Down Expand Up @@ -32,3 +32,6 @@ UEIs are [specifically structured](https://www.gsa.gov/about-us/organization/fed
* SAM.gov requests should be limited
* SAM.gov downtime should be addressed
* SAM.gov entity data may be private and should be limited in what it returns

## Note
Was previously ADR 0007; renamed/renumbered when PDRs and ADRs were merged.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# 8. How to Handle Bulk Data Uploads
# 11. How to Handle Bulk Data Uploads

Date: 2022-05-13

Expand Down Expand Up @@ -46,4 +46,7 @@ The final questions remaining from this issue:
## Consequences

* It will be extra time to work on a practice validator on the site (similar to @jadudm's spike), and possibly should be moved until after the 10/1/2022 launch
* Accepting bulk data uploads is definitely within scope, but determining what "valid" file upload rules will take some time and possibly involve the Census developers to help figure that out
* Accepting bulk data uploads is definitely within scope, but determining what "valid" file upload rules will take some time and possibly involve the Census developers to help figure that out

## Note
Was previously ADR 0008; renamed/renumbered when PDRs and ADRs were merged.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# 4. FAC Migration Plan
# 12. FAC Migration Plan

Introduced: 20220513

Expand Down Expand Up @@ -33,4 +33,7 @@ Provide clarity for the migration plan from the old Census FAC to the new TTS FA
#### Important considerations to reach these goals:

* RISK: We need to retain resources, either on Census side or GSA side, to facilitate transition in April. We are exploring various ways we can keep them on to mitigate this risk.
* Document how every field from the old system will or will not be migrated to and from the old system using the [FAC Field Rosetta Stone](https://docs.google.com/spreadsheets/d/1e-NQPeXk9pcQhA9PEbywkfoJa1bcTunKsVsBjqFYVK4/edit#gid=1955175057) document (internal, not public).
* Document how every field from the old system will or will not be migrated to and from the old system using the [FAC Field Rosetta Stone](https://docs.google.com/spreadsheets/d/1e-NQPeXk9pcQhA9PEbywkfoJa1bcTunKsVsBjqFYVK4/edit#gid=1955175057) document (internal, not public).

## Note
Was previously PDR 0004; renamed/renumbered when PDRs and ADRs were merged.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# 5. Proposed user-visible submission statuses
# 13. Proposed user-visible submission statuses

Introduced: 20220516.
Introduced: 2022-05-16.

## Status

Expand Down Expand Up @@ -104,3 +104,6 @@ My current assumption is that in such cases the submission will be set to `In pr
In the following rough diagram, phases are ellipses and statuses are boxes:

![image](https://user-images.githubusercontent.com/2626258/168630560-16446868-2d4d-425d-8fbc-13839d4e172d.png)

## Note
Was previously PDR 0005; renamed/renumbered when PDRs and ADRs were merged.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# 9. How to document compliance work
# 14. How to document compliance work

Date: 2022-06-02

Expand Down Expand Up @@ -35,3 +35,6 @@ This is one option that provides adequate coverage, leverages one of the few com
* Trestle is still in active development
* OSCAL is still in active development
* An SSP generated by Trestle may need to be converted based on an ISSO's preference

## Note
Was previously ADR 0009; renamed/renumbered when PDRs and ADRs were merged.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# 5. Determine certify audit submission path
# 15. Determine certify audit submission path

Introduced: 20220714.
Introduced: 2022-07-14.

## Status

Expand All @@ -21,4 +21,7 @@ We decided on Option 1 (Special roles/UX in the app to sign digitally) as we bel
* Risk: Could lead to more audit churn if they are uploaded without signatures
* Risk: Could lead to more scanned PDFs being uploaded rather than searchable ones, slowing down auditors

We are moving forward with the caveat that we may consider some lighter-weight email notification system like a mailto link if we think sending emails directly from the app will be too challenging and not useful elsewhere in the app.
We are moving forward with the caveat that we may consider some lighter-weight email notification system like a mailto link if we think sending emails directly from the app will be too challenging and not useful elsewhere in the app.

## Note
Was previously PDR 0006; renamed/renumbered when PDRs and ADRs were merged.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# 10. File Uploads
# 16. File Uploads

Date: 2022-08-01

Expand Down Expand Up @@ -39,3 +39,6 @@ For the frontend portion of this, we will:
- The placement of the file upload component needs to be confirmed or designed. (Check with UX/Design teams)
- The success/failure indicator needs to be confirmed or designed. (Check with UX/Design teams)
- The destination on succcessful upload needs to be confirmed or designed. (Check with UX/Design teams)

## Note
Was previously ADR 0010; renamed/renumbered when PDRs and ADRs were merged.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# 11. Excel Generation and Validation
# 17. Excel Generation and Validation

Date: 2022-08-23

Expand Down Expand Up @@ -49,4 +49,7 @@ The Excel process under discussion would have at least three levels of validatio
7. If a validation rule system (like JSON Schema) was used, it would have to be independent of Django's ORM models. This means updates to one would have to also be applied to the other. If this is not done, they would be out of sync and/or invalid data could be passed back and forth.
8. A validation rule system might also mean needing two schemas, one for a finalized submission and one for an in-progress submission, as we allow parts of the form to be filled out at a time.
9. Testing this would probably mean writing unit tests along with a JSON Schema (or other library) along with example JSON structures that validate both working and not working examples.
10. Testing Excel would probably mean providing some example files that pass and don't pass validation, then using the library and unit tests to ensure the files extract data and match expected results.
10. Testing Excel would probably mean providing some example files that pass and don't pass validation, then using the library and unit tests to ensure the files extract data and match expected results.

## Note
Was previously ADR 0011; renamed/renumbered when PDRs and ADRs were merged.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# 6. Proposed UEI handling change
# 18. Proposed UEI handling change

Introduced: 2022-09-02

Expand Down Expand Up @@ -100,3 +100,6 @@ If a verified UEI is not present, finalization will require the user to attest t
Organizations are required to submit these reports, and it's not currently clear that the FAC would be within its rights to refuse to let a submission proceed without a UEI—especially if the submission lacks a UEI because the organization has for some reason been unable to acquire one.

In order to determine product and technical direction on how to handle this case, we first need policy direction.

## Note
Was previously PDR 0007; renamed/renumbered when PDRs and ADRs were merged.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# 11. Use the Static Site Generator Eleventy for the FAC Frontend
# 19. Use the Static Site Generator Eleventy for the FAC Frontend

Date: 2022-09-06

Expand Down Expand Up @@ -27,3 +27,6 @@ We will use a combination of Eleventy's [is-land plugin](https://is-land.11ty.de

* Future components will be able to leverage Alpine, requiring less custom JS.
* Adding `is-land` will allow the team to add other libraries on a page-by-page basis if needed.

## Note
Was previously ADR 0011; renamed/renumbered when PDRs and ADRs were merged.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# 11. Store Question Copy as JSON
# 20. Store Question Copy as JSON

Date: 2022-09-15

Expand Down Expand Up @@ -38,4 +38,7 @@ Internally, each file will contain a single JSON dictionary, which maps a stable
## Consequences

- The JSON files containing the question copy for each year will be the source of truth for this data, though it will likely appear elsewhere in the code base. Specifically, the question copy may be included as part of the validation schema for audit submissions. The generation of these schemas should be configured to automatically pull the question copy from these JSON files to avoid having to keep multiple question sources synchronized.
- Question IDs should remain the same if/when the copy associated with the question changes. This is what will allow us to merge conceptually equivalent fields across years in a later step, facilitating a better search & presentation experience for public audit report access.
- Question IDs should remain the same if/when the copy associated with the question changes. This is what will allow us to merge conceptually equivalent fields across years in a later step, facilitating a better search & presentation experience for public audit report access.

## Note
Was previously ADR 0011; renamed/renumbered when PDRs and ADRs were merged.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# 12. Switch to a unified Django web application architecture
# 21. Switch to a unified Django web application architecture

Date: 2022-10-28

Expand Down Expand Up @@ -57,3 +57,6 @@ We will move the current frontend into Django, use Django to manage state, add w
## Updated system diagram

![image](https://raw.githubusercontent.com/GSA-TTS/FAC/main/docs/architecture/diagrams/FAC_System_Context_unified.png)

## Note
Was previously ADR 0012; renamed/renumbered when PDRs and ADRs were merged.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# 13. Use Infrastructure-as-Code (IaC) for environment access
# 22. Use Infrastructure-as-Code (IaC) for environment access

Date: 2022-12-09

Expand Down Expand Up @@ -30,3 +30,5 @@ The process needs to be structured, well-documented, and auditable for complianc
* People can come and go without the process changing.
* Single-sources environment access approvals with repository access approvals.

## Note
Was previously ADR 0013; renamed/renumbered when PDRs and ADRs were merged.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# 14. Store submission data as JSON
# 23. Store submission data as JSON

Date: 2023-01-04

Expand All @@ -8,7 +8,7 @@ Accepted

## Context

We need to store the data that users submit as part of their Single Audit Checklist packages. The storage mechanism needs to accomodate the following:
We need to store the data that users submit as part of their Single Audit Checklist packages. The storage mechanism needs to accommodate the following:
- The Single Audit Checklist schema can (and in fact does) change over time. These changes include the addition of new fields, the removal of deprecated fields, and adjustments to the field validation logic.
- The FAC needs to maintain the ability to accept and validate backdated submissions, using the schema and validation logic for the submission year (i.e. not the most recent schema/validation version).
- Users can choose to submit their audit package through a web form or by uploading an Excel file.
Expand All @@ -23,3 +23,6 @@ We will partition the Single Audit Checklist data model into high-level sections
* Schema and validation rules for sections stored as JSON will be portable to other systems
* Increased code base complexity resulting from schema and validation rules living outside the object-relational mapper
* Additional burden on developers to be familiar with JSON Schema

## Note
Was previously ADR 0014; renamed/renumbered when PDRs and ADRs were merged.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# 15. Store uploaded Excel files
# 24. Store uploaded Excel files

Date: 2023-02-06

Expand Down Expand Up @@ -58,3 +58,6 @@ Last-write-wins here also applies to the submission data we store in our databas
- We will need to add file upload and download to our Django views, and handle the above validation logic.
- We will need to add the `ClamAV` functionality.
- In the future, if we insist on storing Excel files, we will need to follow various retention policy requirements regarding them.

## Note
Was previously ADR 0015; renamed/renumbered when PDRs and ADRs were merged.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# 16. How to document compliance work
# 25. How to document compliance work

Date: 2023-02-27

Expand All @@ -10,7 +10,7 @@ Accepted

Compliance documentation is an essential component of working in the open while providing clear and obvious evidence of security considerations, concerns, and mitigations.

The [prior decision](https://github.com/GSA-TTS/FAC/blob/main/docs/architecture/decisions/0009-compliance-documentation.md) to use [OSCAL](https://pages.nist.gov/OSCAL/) with [Trestle](https://github.com/IBM/compliance-trestle) isn’t practical given personnel considerations.
The [prior decision](https://github.com/GSA-TTS/FAC/blob/main/docs/architecture/decisions/0014-compliance-documentation.md) to use [OSCAL](https://pages.nist.gov/OSCAL/) with [Trestle](https://github.com/IBM/compliance-trestle) isn’t practical given personnel considerations.

## Decision

Expand All @@ -28,3 +28,7 @@ We will continue to [generate diagrams with PlantUML](https://github.com/GSA-TTS

* ATO documentation will need to be tracked using documents/spreadsheets, requiring manual coordination.
* We will remove the `/compliance` directory from `main`.


## Note
Was previously ADR 0016; renamed/renumbered when PDRs and ADRs were merged.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# 17. SAM.gov integration for MVP
# 26. SAM.gov integration for MVP

Date: 2023-03-09

Expand Down Expand Up @@ -42,3 +42,6 @@ There are data cleanliness consequences. To mitigate this, we are able to put in
There are operational considerations. To mitigate this, we will revisit SAM.gov uptime and the state of UEI transition at a later point, and revisit the question of how to handle UEIs that cannot be validated at submission-time. However, this operational posture is in keeping with the AY22 submission, and does not represent a regression from the C-FAC to G-FAC.

There are paths to iterative improvement. As systems improve, we will be able to iteratively improve our system and its interaction with others. This pathway reduces user burden now, and leaves pathways for systemic improvement through future revisions.

## Note
Was previously ADR 0017; renamed/renumbered when PDRs and ADRs were merged.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# 1. Data validation schema in Jsonnet
# 27. Data validation schema in Jsonnet

Date: 2023-04-10

Expand Down Expand Up @@ -40,3 +40,5 @@ This adds another tool to our backend pipeline, and a dependancy that could brea

There are no security implications that we are aware of from use of this tool as a compiler; no more than using any other compiler, that is. This tooling is part of our build process, and is not exposed as part of the user-facing application.

## Note
Was previously ADR 0018; renamed/renumbered when PDRs and ADRs were merged.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# UEI required for submission
# 28. UEI required for submission

Introduced: 2023-05-31

Expand Down Expand Up @@ -41,3 +41,5 @@ Put simply: it will not be possible to submit the SF-SAC without a valid UEI. A
2. SAM.gov is online and available.
3. The SAM.gov API indicates that the UEI presented is valid and in the SAM.gov database

## Note
Was previously PDR 0008; renamed/renumbered when PDRs and ADRs were merged.