Merge pull request #147 from reitermb/main

Add Flat File Metadata code doc, Update design guide & research docs, #502, #503
raft-tech · Jan 27, 2021 · c2e9d2e · c2e9d2e
2 parents 634d744 + cabdb0e
commit c2e9d2e
Show file tree

Hide file tree

Showing 7 changed files with 350 additions and 70 deletions.
diff --git a/docs/Background/Current-TDRS.md b/docs/Background/Current-TDRS.md
@@ -96,7 +96,7 @@ If sending sample data, a report on the stratum totals for SSP-MOE reporting.
 ## Additional resources:
 * [Instructions and form templates for all sections](https://www.acf.hhs.gov/ofa/resource/policy/pi-ofa/2008/200809/tanf-acf-pi-2008-07)
 * [Legacy screenshots are in the process map](https://app.mural.co/t/officeoffamilyassistance2744/m/gsa6/1593114727729/906879aaeb5467c27f0ae3dfbcf5fcfd9cf8ca89)
-* [More about TDRS users](../User-Research/User-Types.md)
+* [More about TDRS users](../User-Research/Stakeholders-and-Personas.md)
 * [Tribal TANF Data Coding Instructions](https://www.acf.hhs.gov/ofa/resource/tribal-tanf-data-coding-instructions)
 
 
diff --git a/docs/Design/Design-Guide.md b/docs/Design/Design-Guide.md
@@ -1,26 +1,75 @@
-## Content guide
+# Design guide
+
+**Table of Contents:**
+
+- [Content Guide](#content-guide)
+- [Accessibility](#accessibility)
+- [Visual styling](#visual-styling)
+- [Figma artifacts](#figma-artifacts)
+
+---
+
+# Content guide
+
 We're continuing to learn about the content needs of the new TDRS. Future teams should plan to write in plain language and continually test site navigation and comprehension.
 
 **Resources:**
+
 * [ACF content guide](https://www.acf.hhs.gov/digital-toolbox/content)
 * [plainlanguage.gov](https://plainlanguage.gov/)
 * [18F content guide](https://content-guide.18f.gov/)
 
-## Accessibility
-Everyone who works on government websites has a role to play in making federal resources accessible and inclusive. The new TDRS should meet [508 accessibility standards](https://www.section508.gov/) and use accessible design and development best practices.
+---
+
+# Accessibility
+
+Everyone who works on government websites has a role to play in making federal resources accessible and inclusive. The new TDRS should meet 508 accessibility standards and use accessible design and development best practices.
 
 **Resources:**
+
 * [Accessibility for teams](https://accessibility.digital.gov/)
 * [18F Accessibility guide](https://accessibility.18f.gov/)
 * [Section508.gov](https://www.section508.gov/)
 
-## Visual styling
+---
+
+# Visual styling
+
+## ACF styles
 
-### ACF styles
 Information on existing ACF styles can be found in the [ACF Digital Toolbox](https://www.acf.hhs.gov/digital-toolbox)
-* [Colors](https://www.acf.hhs.gov/digital-toolbox/visuals/visual-style-guide)
-* [Typography](https://www.acf.hhs.gov/digital-toolbox/visuals/typography-and-fonts)
+* [ACF Colors](https://www.acf.hhs.gov/digital-toolbox/visuals/visual-style-guide)
+
+* [ACF Typography](https://www.acf.hhs.gov/digital-toolbox/visuals/typography-and-fonts)
+
+
+
+## U.S. Web Design System (USWDS)
+We'll be using  [USWDS](https://designsystem.digital.gov/), a design system for the federal government, for the new TANF Data Portal (TDP). USWDS provides us with UX guidance and code for [common UI components](https://designsystem.digital.gov/components/) as well as [guidance on how to comply with the 21st Century Integrated Digital Experiences Act (IDEA)](https://designsystem.digital.gov/website-standards/?dg).
+
+---
+
+# Figma artifacts
+
+## User flows
+
+To improve our workflow in ideating and supporting development, we created a [user flows page](<https://www.figma.com/file/irgQPLTrajxCXNiYBTEnMV/TDP-Mockups-For-Feedback?node-id=1277%3A8087>) in Figma to map and explore user actions and outcomes in a modular environment that can be easily changed and extended. 
+
+This living document covers actions a user would take, system actions that happen on the backend, and destinations that are reached after actions are executed. To communicate how these actions and outcomes intertwine, we included primary and secondary paths that a user can take within the flow. These flows have also proven useful for certain security / compliance workflow documentation needs.
+
+## Mockups & prototypes
+
+[Full Site](https://www.figma.com/file/irgQPLTrajxCXNiYBTEnMV/TDP-Mockups-For-Feedback?node-id=0%3A1>) - A visual site map page encompassing all reviewed dev-ready mockups that have been delivered.
+
+**Reviewed**
+
+-  [Upload and Download Data Reports](<https://www.figma.com/file/irgQPLTrajxCXNiYBTEnMV/TDP-Mockups-For-Feedback?node-id=2933%3A0>) associated to issues [#427](https://github.com/raft-tech/TANF-app/issues/427) and [#415](https://github.com/raft-tech/TANF-app/issues/415)
+-  [User Management (Admin tools)](<https://www.figma.com/file/irgQPLTrajxCXNiYBTEnMV/TDP-Mockups-For-Feedback?node-id=2441%3A12231>) associated to issues [#404](https://github.com/raft-tech/TANF-app/issues/404), [#327](https://github.com/raft-tech/TANF-app/issues/327), and [#164](https://github.com/raft-tech/TANF-app/issues/164)
+-  [Reset Password](<https://www.figma.com/file/irgQPLTrajxCXNiYBTEnMV/TDP-Mockups-For-Feedback?node-id=2933%3A0>) associated to issue [#318](https://github.com/raft-tech/TANF-app/issues/318)
+-  [Concept Prototype (For round 3 research)](<https://www.figma.com/file/irgQPLTrajxCXNiYBTEnMV/TDP-Mockups-For-Feedback?node-id=1381%3A0>) associated to issues [#275](https://github.com/raft-tech/TANF-app/issues/275) and [#115](https://github.com/raft-tech/TANF-app/issues/115)
+-  [Responsive Mobile Designs (Admin pages)](<https://www.figma.com/file/irgQPLTrajxCXNiYBTEnMV/TDP-Mockups-For-Feedback?node-id=618%3A14>) associated to issue [#178](https://github.com/raft-tech/TANF-app/issues/178)
+-  [Request Access (New user)](<https://www.figma.com/file/irgQPLTrajxCXNiYBTEnMV/TDP-Mockups-For-Feedback?node-id=628%3A468>) associated to issue [#166](https://github.com/raft-tech/TANF-app/issues/166)
 
-### USWDS
-We'll be using the [U.S. Web Design System (USWDS)](https://designsystem.digital.gov/), a design system for the federal government, for a new TDRS system. USWDS provides us with UX guidance and code for [common UI components](https://designsystem.digital.gov/components/) and also [guidance on how to comply with the 21st Century Integrated Digital Experiences Act (IDEA)](https://designsystem.digital.gov/website-standards/?dg)
+**In Backlog**
 
+- [Ideas on Validation](<https://www.figma.com/file/irgQPLTrajxCXNiYBTEnMV/TDP-Mockups-For-Feedback?node-id=1412%3A9795>) associated to issue [#295](https://github.com/raft-tech/TANF-app/issues/295)
diff --git a/docs/Product-Strategy/Vision-and-Stakeholders.md b/docs/Product-Strategy/Vision-and-Stakeholders.md
@@ -15,4 +15,4 @@ We've developed a [stakeholder map](https://app.mural.co/t/officeoffamilyassista
 
 ![A map of the different stakeholders who are affected by TDRS](https://github.com/HHS/TANF-app/blob/master/design-assets/research-artifacts/TDRS-stakeholder-map.png)
 
-We've included more detail about TDRS audiences in on our [user types page](../User-Research/User-Types.md).
+We've included more detail about TDRS audiences in on our [user types page](../User-Research/Stakeholders-and-Personas.md).
diff --git a/docs/User-Research/Flat File Metadata Guide.md b/docs/User-Research/Flat File Metadata Guide.md
@@ -0,0 +1,144 @@
+# Flat File Metadata Guide
+
+[Issue #248](https://github.com/raft-tech/TANF-app/issues/248)
+
+---
+
+**Table of Contents:**
+
+- [Background](#background)
+- [Header structure](#header-structure)
+- [Trailer strcture](#trailer-structure)
+- [Flat file naming conventions](#flat-file-naming-conventions)
+- [Other notes on flat file conventions](#other-notes-on-flat-file-conventions)
+- [Metadata validation definitions](#metadata-validation-definitions)
+- [MVP metadata validation](#mvp-metadata-validation)
+
+---
+
+# Background
+
+During OFA research, we discovered that submitted flat files have header and trailer errors. Further research is needed to understand how the header and trailer are created for flat files, what is the cause of the errors, and what information can be automatically added (by default), and what information needs to be added by STTs. Having this understanding would allow us to standardize header and trailers and gather the correct information from STTs when creating flat files for submission.
+
+---
+
+# Header structure
+
+Example header:
+
+`HEADER20152G02000TAN2 N`
+
+| **Header Field**                           | **Excerpt / Decoding**                                       |
+| :----------------------------------------- | :----------------------------------------------------------- |
+| <br />**Title**<br /><br />                | <br />`HEADER` <br /><br />                                  |
+| <br />**Calendar Quarter**<br /><br />     | <br />`20152` (Quarter 2 of 2015)<br /><br />                |
+| <br />**Data Type**<br /><br />            | <br />`G` (Aggregate, i.e, TANF or SSP Report Section 3)<br /><br /> |
+| <br />**State Fips Code**<br /><br />      | <br />`02` (Alaska)<br /><br />                              |
+| <br />**Tribe Code**<br /><br />           | <br />`000` (This is equivalent to N/A, in this case the header refers only to Alaska as a state, not a tribe in Alaska)<br /><br /> |
+| <br />**Program Type**<br /><br />         | <br />`TAN` (TANF Report, as opposed to SSP-MOE which would be coded as 'SSP')<br /><br /> |
+| <br />**Edit Indicator**<br /><br />       | <br />`2` (Legacy data point that seems to refer to fTANF validation. '2' returns only Fatal Edits whereas '1' would return Warning Edits AND Fatal Edits)<br /><br /> |
+| <br />**Encryption Indicator**<br /><br /> | <br />`Blank Space` (A blank means that social security numbers are NOT encrypted, 'E' in place of the blank would mean that they were encrypted)<br /><br /> |
+| <br />**Update Indicator**<br /><br />     | <br />`N` (New Data, as opposed to Delete and Replace 'D', or Add and Update 'U'. Non 'N' values come into play during resubmission)<br /><br /> |
+
+![Screenshot of OFA header metadata decodimg](https://user-images.githubusercontent.com/22622384/104673247-562d3180-56af-11eb-9930-e6b758b10383.png)
+
+---
+
+# Trailer structure
+
+Example trailer:
+
+`TRAILER0000042[9 blank spaces here]`
+
+| **Trailer Field**                       | **Excerpt / Decoding**                                       |
+| :-------------------------------------- | :----------------------------------------------------------- |
+| <br />**Title**<br /><br />             | <br />`TRAILER` <br /><br />                                 |
+| <br />**Number of Records**<br /><br /> | <br />`0000042` (Number of records in report in a seven-digit format)<br /><br /> |
+| <br />**Blank Spaces**<br /><br />      | <br />` ` (9 blank spaces)<br /><br />                       |
+
+
+
+---
+
+# Flat file naming conventions
+
+ADS.E2J. Does not change across report types or transmission methods. The third decimal delineated section (FTP# or NDM#) identify transmission method (SFTP vs Cyberfusion) and TANF or SSP-MOE report section in the file. The fourth section Identifies Program Type (TANF vs SSP-MOE as ‘TS’ or ‘MS’ respectively) followed by State Fips Code. In the case of Tribes, only TANF reports (TS) are submitted and a 3 digit Tribal Code follows.
+
+For states and territories, ## represents their 2 digit Fips Code
+
+| **Program** | **Transmission Method** | **Report Section** | **File Naming Convention** |
+| :---------- | :---------------------- | :----------------- | :------------------------- |
+| TANF        | SFTP                    | 1 (Active)         | `ADS.E2J.FTP1.TS##`        |
+| TANF        | SFTP                    | 2 (Closed)         | `ADS.E2J.FTP2.TS##`        |
+| TANF        | SFTP                    | 3 (Aggregate)      | `ADS.E2J.FTP3.TS##`        |
+| TANF        | SFTP                    | 4 (Stratum)        | `ADS.E2J.FTP4.TS##`        |
+| TANF        | CyberFusion             | 1 (Active)         | `ADS.E2J.NDM1.TS##`        |
+| TANF        | CyberFusion             | 2 (Closed)         | `ADS.E2J.NDM2.TS##`        |
+| TANF        | CyberFusion             | 3 (Aggregate)      | `ADS.E2J.NDM3.TS##`        |
+| TANF        | CyberFusion             | 4 (Stratum)        | `ADS.E2J.NDM4.TS##`        |
+| SSP         | SFTP                    | 1 (Active)         | `ADS.E2J.FTP1.MS##`        |
+| SSP         | SFTP                    | 2 (Closed)         | `ADS.E2J.FTP2.MS##`        |
+| SSP         | SFTP                    | 3 (Aggregate)      | `ADS.E2J.FTP3.MS##`        |
+| SSP         | SFTP                    | 4 (Stratum)        | `ADS.E2J.FTP4.MS##`        |
+| SSP         | CyberFusion             | 1 (Active)         | `ADS.E2J.NDM1.MS##`        |
+| SSP         | CyberFusion             | 2 (Closed)         | `ADS.E2J.NDM2.MS##`        |
+| SSP         | CyberFusion             | 3 (Aggregate)      | `ADS.E2J.NDM3.MS##`        |
+| SSP         | CyberFusion             | 4 (Stratum)        | `ADS.E2J.NDM4.MS##`        |
+
+For tribes, ### represents their 3 digit Tribe Code
+
+| **Report Section** | **File Naming Convention** |
+| :----------------- | :------------------------- |
+| 1 (Active)         | `ADS.E2J.FTP1.TS###`       |
+| 2 (Closed)         | `ADS.E2J.FTP2.TS###`       |
+| 3 (Aggregate)      | `ADS.E2J.FTP3.TS###`       |
+
+---
+
+# Other notes on flat file conventions
+
+- Sections 1-4 of a TANF or SSP-MOE report all have their own flat files. The Header and Trailer are a part of each, but should remain consistent with the exception of the Data Type field.
+- Fiscal-year reports are submitted with multiple headers and trailers in a single text file, each representing their respective quarter. 
+
+---
+
+# Metadata validation definitions
+
+**Automatic Validation**: Validation that can happen *without* a user manually inputting metadata. This includes values associated to users (e.g. Fips and Tribal codes), user actions inherently tied to metadata (e.g. An action labeled as “Create TANF Report for Q1 of 2020” as opposed to a blank slate style “New Report”), and properties that do not change (e.g. Title fields of the header and trailer). 
+
+
+
+**Guardrailed Validation**: Validation driven by comparisons against data manually entered by users (e.g. Dropdowns for selecting Quarter and Year being compared to the Calendar Quarter header field.)
+
+
+
+🔍 **Research**: There are open questions and we’d like to base a recommendation on insights from Phase III Research ([October STT Research](https://github.com/HHS/TANF-app/blob/main/docs/User-Research/2020%2C%20Fall%20-%20Understanding%20STT%20Roles%2C%20Source%20of%20Truth%2C%20and%20Metadata.md)) and continued solutions ideation (Both design and technical).
+
+---
+
+# MVP metadata validation
+
+Below is an overview of which Header and Trailer properties will have automatic validation, which will be guardrailed, and which require further research. We’ve differentiated here between what’s currently illustrated in the Concept Prototype* and what we evaluated as being technically feasible.
+
+*Note that the [concept prototype](https://www.figma.com/proto/y15co5xc7MIZXBnWBOF6hJ/Conceptual-Mockups-for-STT-Feedback?node-id=9%3A332&viewport=187%2C385%2C0.11285778135061264&scaling=min-zoom) may be iterated upon once the Conversation Guide is complete. The delta between what’s feasible and what’s being illustrated in the Concept Prototype *may* narrow. 
+
+| **Header Field**                           | **Automatic <br />(Concept Prototype)** | **Guardrailed<br /> (Concept Prototype)** | **Automatic <br />(Technically Feasible)** | **Guardrailed <br />(Technically Feasible)** |
+| :----------------------------------------- | :-------------------------------------- | :---------------------------------------- | :----------------------------------------- | :------------------------------------------- |
+| <br />**Title**<br /><br />                | <br />✓ Yes<br /><br />                 | <br /><br /><br />                        | <br />✓ Yes<br /><br />                    | <br /><br /><br />                           |
+| <br />**Calendar Quarter**<br /><br />     | <br /><br /><br />                      | <br />✓ Yes<br /><br />                   | <br />✓ Yes<br /><br />                    | <br /><br /><br />                           |
+| <br />**Data Type**<br /><br />            | <br />✓ Yes<br /><br />                 | <br /><br /><br />                        | <br />✓ Yes<br /><br />                    | <br /><br /><br />                           |
+| <br />**State Fips Code**<br /><br />      | <br />✓ Yes<br /><br />                 | <br /><br /><br />                        | <br />✓ Yes<br /><br />                    | <br /><br /><br />                           |
+| <br />**Tribe Code**<br /><br />           | <br />✓ Yes<br /><br />                 | <br /><br /><br />                        | <br />✓ Yes<br /><br />                    | <br /><br /><br />                           |
+| <br />**Program Type**<br /><br />         | <br /><br /><br />                      | <br />✓ Yes<br /><br />                   | <br />✓ Yes<br /><br />                    | <br /><br /><br />                           |
+| <br />**Edit Indicator**<br /><br />       | <br />🔍 Research<br /><br />            | <br />🔍 Research<br /><br />              | <br />🔍 Research<br /><br />               | <br /><br /><br />                           |
+| <br />**Encryption Indicator**<br /><br /> | <br /><br /><br />                      | <br /><br /><br />                        | <br />🔍 Research<br /><br />               | <br /><br /><br />                           |
+| <br />**Update Indicator**<br /><br />     | <br />✓ Yes (For New)<br /><br />       | <br /><br /><br />                        | <br />✓ Yes (For New)<br /><br />          | <br />🔍 Research<br /><br />                 |
+
+| **Trailer Field**                       | **Automatic <br />(Concept Prototype)** | **Guardrailed <br />(Concept Prototype)** | **Automatic <br />(Technically Feasible)** | **Guardrailed <br />(Technically Feasible)** |
+| :-------------------------------------- | :-------------------------------------- | :---------------------------------------- | :----------------------------------------- | :------------------------------------------- |
+| <br />**Title**<br /><br />             | <br />✓ Yes<br /><br />                 | <br /><br /><br />                        | <br />✓ Yes<br /><br />                    | <br /><br /><br />                           |
+| <br />**Number of Records**<br /><br /> | <br />✓ Yes<br /><br />                 | <br /><br /><br />                        | <br />✓ Yes<br /><br />                    | <br /><br /><br />                           |
+| <br />**Blank Spaces**<br /><br />      | <br />✓ Yes<br /><br />                 | <br /><br /><br />                        | <br />✓ Yes<br /><br />                    | <br /><br /><br />                           |
+
+---
+
Original file line number	Diff line number	Diff line change
Expand Up		@@ -15,4 +15,4 @@ We've developed a [stakeholder map](https://app.mural.co/t/officeoffamilyassista

		![A map of the different stakeholders who are affected by TDRS](https://github.com/HHS/TANF-app/blob/master/design-assets/research-artifacts/TDRS-stakeholder-map.png)

		We've included more detail about TDRS audiences in on our [user types page](../User-Research/User-Types.md).
		We've included more detail about TDRS audiences in on our [user types page](../User-Research/Stakeholders-and-Personas.md).