You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Validation has its own logic for reading and checking CSVs. This is now redundant to utils.read_csv_to_dataframe and less comprehensive. It would make sense to read CSVs for validation in the same way that we're going to read them in execute.
Currently, CSV validation check that:
the csv exists
the file has the necessary permissions
exact column names exist in the csv
With utils.read_csv_to_dataframe, we can also easily check that:
at least one column name matching a pattern exists in the csv
values can be cast to the appropriate data type for each column
related issues
This makes sense to do before #327 because read_csv_to_dataframe handles path expansion.
I'd also like to revisit whether we can drop support for Excel files (which is only allowed in one table in HRA) rather than writing additional code to support them.
The text was updated successfully, but these errors were encountered:
Validation has its own logic for reading and checking CSVs. This is now redundant to
utils.read_csv_to_dataframe
and less comprehensive. It would make sense to read CSVs for validation in the same way that we're going to read them inexecute
.Currently, CSV validation check that:
With
utils.read_csv_to_dataframe
, we can also easily check that:related issues
This makes sense to do before #327 because
read_csv_to_dataframe
handles path expansion.I'd also like to revisit whether we can drop support for Excel files (which is only allowed in one table in HRA) rather than writing additional code to support them.
The text was updated successfully, but these errors were encountered: