-
Notifications
You must be signed in to change notification settings - Fork 64
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement bracketed unit parsing in forcing CSV headers #511
Conversation
fccdd9a
to
33cd8fe
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Other than testing the case mentioned, the implementation looks reasonable to me.
@@ -164,3 +176,63 @@ TEST_F(CsvPerFeatureForcingProviderTest, TestGetAvailableForcingOutputs) | |||
|
|||
} | |||
|
|||
///Test CSV Units Parsing |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Might be good to add a test for extra characters after the unit brackets, e.g. var (unit) extra, ...
if (var_name_open != std::string::npos) { | ||
// found matching opening bracket/parenth | ||
|
||
units = var_name.substr(var_name_open + 1); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My only concern is what happens with extra characters after the closing bracket/paren.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
are we expecting anything after a unit declaration? My understanding was that unit declarations would be required to be at the end, or would not be parsed at all (see the outer if
construct)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not expecting it, but should be prepared...a warning, error, or something... Would not catching that case cause some weird behavior in the units that we don't expect or can't track down easily?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we add a check after the well-known checks to check if units
is still empty, and throw a warning then? This would then tell us/the user that the column that threw the warning has no unit mapping (i.e. like how UnitsHelper::get_converted_value
throws a warning when a mapping doesn't exist).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The issue was originally written to treat this case as brackets that do not represent a unit.... in order for the unit to be recognized it would have to be the last thing in the string. We can revisit if we want.
The thought was that there might be (for some reason??) column names with another set of brackets representing something else. In that case, if they were not at the end, then the brackets and their contents would be ignored by the unit parser. If they were at the end, then the workaround could be to add an extra set of different brackets to the end, which would either have valid units or be empty and picked up as an empty units string (didn't cover this case in the issue though, probably).
729e70a
to
4b9114a
Compare
4b9114a
to
56f7a98
Compare
Resolves #510 by implementing unit parsing within brackets or parentheses appended to column names in forcing CSV headers.
Additions
CsvPerFeatureForcingProvider::read_csv
.Testing
Todos
Checklist
Target Environment support