-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
VDYP-168 Validate FIPStart input data #19
Merged
Merged
Changes from 1 commit
Commits
Show all changes
111 commits
Select commit
Hold shift + click to select a range
2041ae0
Control file parser
smithkm 37e085b
Docs, new test, refinements to configuring control file parser
smithkm dcb48f2
Write control file
smithkm a410ac7
Parsing Fip control file
smithkm a08c0a0
Progress on parsing data files
smithkm 92487ac
Rewrite control file parser to use LineParser
smithkm 262164d
BecParserTest
smithkm 3e93b27
Docs and refinements
smithkm 1aa2dd1
FIP Control parsing
smithkm b48d5b4
Merge branch 'main' of github.com:bcgov/VDYP
smithkm 93c6554
Read SP0 definitions
smithkm 4b017f7
Volume Equation Group Parsing
smithkm 876e13e
VolumeEquationGroupParser
smithkm c763600
Move control key for volume eqn group to parser
smithkm 2a5763c
Decay and breakage groups
smithkm f2ae928
Validate that SP0-BEC->Group ID mappings are complete
smithkm dad77be
Parse equation modifier data file
smithkm dae7cd9
Parse stocking factor
smithkm 34f165f
Site Curve parsing and ran formatter
smithkm 4b1a0fb
Site curve age max loading
smithkm e326fb8
CoefficientParser
smithkm 7d1f30e
Upper Coeficcient parser, and fixes for regular coefficient parser
smithkm 6e4741a
HL coefficient parser
smithkm 45f7be2
Nonprimary coefficient parser
smithkm a591af7
Run formattter
smithkm faf23ca
BySpeciesDqCoefficientParser
smithkm 3e10ac3
Componetn size limit
smithkm 40f3a5d
UtilComponentBaseAreaParser
smithkm 14faf65
Stubbing FIP parser methods
smithkm 72ca5aa
Formatter
smithkm c3fbe42
Document behaviour of Fortran parsers
smithkm 101d68c
CloseUtilVolumeParser
smithkm 7adc378
VolumeNetDecayParser
smithkm 1a45dc6
Small component parsers
smithkm e4a9636
Add tests for FIP control file parser methods
smithkm ca636be
TotalStandWholeStemParser
smithkm 98c60d8
Add nohup log to .gitignore
smithkm b5e9723
Use new hasSpecificEtry matcher for FIP parser unit tests to make err…
smithkm 1575300
UtilComponentWSVolumeParser
smithkm 1d7a178
VolumeNetDecayWasteParser
smithkm e9fa782
BreakageParser
smithkm 6793922
VeteranLayerVolumeAdjustParser
smithkm 0418341
VeteranBQParser
smithkm 7c9fe31
UtilComponentDQParser
smithkm e0e0333
VeteranDQParser
smithkm 33eea60
Refactor to allow for control map access in Line/Value parsers
smithkm 931f9fb
Working
smithkm aad4b5b
Merge remote-tracking branch 'origin/main' into feature/VDYP-120
smithkm bf359ef
Refactor to allow parsers access to the control map
smithkm 2f25e4f
Have all parsers for resources loaded by FIP control file implement C…
smithkm 7644655
Implement BEC index substitution on lookup
smithkm e065c77
Stub for modifier parser
smithkm ff21e3a
Handle missing control file
smithkm 5881877
Set default modifiers for BA and DQ
smithkm 8274efe
Defaults for decay and waste modifiers
smithkm c0e9bb0
Clean up tests
smithkm 86e8396
Run formatter
smithkm c509b29
Apply single species modifier to BA/DQ
smithkm d695eb2
Apply BA/DQ mod to all species
smithkm 4615bc8
Remap jprogram index
smithkm da08fbc
Apply modifiers to veteran BQ coefficients
smithkm e8302b5
HL DEcay and Waste modifiers
smithkm 4fe31e7
Apply HL Modifiers for P1, P2, P3
smithkm 141b375
Nonprimary HL modifiers
smithkm 7207729
Refactor LineParser to allow for streaming parsers to be built on top…
smithkm e86784d
Parser for polygon data file
smithkm cc3b86b
Working on parser for polygon data
smithkm 01af9f7
Parsing polygon data file
smithkm 1d7a0f3
Load fip polygon datafile from control file
smithkm 8415f68
Remove unused imports
smithkm 3660fd2
Fix raw type warning in test
smithkm c84e10b
Fix unused variable warings in unit tests
smithkm 562c641
Remove BecModifier as it is not needed
smithkm 6f07143
Merge branch 'feature/VDYP-120' into feature/VDYP-164
smithkm bad1ba0
Fix unused imports
smithkm f4a615a
Class to replicate type union
smithkm 98cb956
Loading FIP layer data file
smithkm 6abc2d2
Parsing layer-species data file
smithkm 804bf60
Ignore layers with height or crown closure <=0
smithkm 0faa7ef
Processing FIPStart data
smithkm c5e1f50
Validate that polygons always have a primary layer
smithkm b08ac25
Reduce unit test boilerplate
smithkm 5d1674c
Validate the primary layer has YTBH less than total age
smithkm 8e36f1a
Reduce test boilerplate and implement validation of minimum height
smithkm b28e67c
Test for species file not matching polygon file
smithkm 930bb2f
Validate polygon mode is not FIPYOUNG
smithkm 638d32e
Validate years to breast height minimum
smithkm f22209d
TODO note about non-primary validation
smithkm 96fdf8d
Validate minimum site index
smithkm 7f4f678
Validate species for a layer sum to 100%
smithkm 829f211
Species fraction computation to match VDYP7
smithkm 681d7f0
Document Relationship between polygon/layer/species objects and their…
smithkm e690340
Run formatter
smithkm 48cdf3a
Merge branch 'main' into feature/VDYP-120
smithkm c0c9d23
Merge branch 'feature/VDYP-120' into feature/VDYP-164
smithkm 04f0d14
Merge branch 'feature/VDYP-164' into feature/VDYP-168
smithkm f94257e
Prevent shallow clone so SonarCloud can work correctly.
smithkm b72a0ed
More unit tests to improve coverage
smithkm 216d5cd
Change unit test scopes
smithkm abbea98
Remove redundant tests
smithkm d95b4b8
Merge branch 'feature/VDYP-163-VDYP-165' into feature/VDYP-164
smithkm 5792655
Remove redundant traversal of key set
smithkm a5ffcd4
Fix warnings in species parser and grouping parser
smithkm d6b7b71
Merge branch 'feature/VDYP-164' into feature/VDYP-168
smithkm b6f5e15
Constuctors of abstract classes should be protected
smithkm e61c9a5
Replace .collect(Collectors.toList()) with .toList()
smithkm d6064d2
Rename Layer to LayerType
smithkm 2b0c4c0
Unit test for enumParser
smithkm 07f2f29
Value or Marker parser test
smithkm ada7a26
Layer type parser test
smithkm b310f10
Unit tests for map parser and better error messages
smithkm File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -317,6 +317,172 @@ public void testPolygonWithModeFipYoung() throws Exception { | |
|
||
} | ||
|
||
@Test | ||
public void testOneSpeciesLessThan100Percent() throws Exception { | ||
|
||
var polygonId = polygonId("Test Polygon", 2023); | ||
var layer = Layer.PRIMARY; | ||
|
||
testWith( | ||
Arrays.asList(getTestPolygon(polygonId, valid())), // | ||
Arrays.asList(layerMap(getTestPrimaryLayer(polygonId, valid()))), // | ||
Arrays.asList(Collections.singletonList(getTestSpecies(polygonId, layer, x -> { | ||
x.setPercentGenus(99f); | ||
}))), // | ||
app -> { | ||
|
||
var ex = assertThrows(ProcessingException.class, () -> app.process()); | ||
|
||
assertThat( | ||
ex, | ||
hasProperty( | ||
"message", | ||
is( | ||
"Polygon " + polygonId | ||
+ " has PRIMARY layer where species entries have a percentage total that does not sum to 100%." | ||
) | ||
) | ||
); | ||
} | ||
); | ||
|
||
} | ||
|
||
@Test | ||
public void testOneSpeciesMoreThan100Percent() throws Exception { | ||
|
||
var polygonId = polygonId("Test Polygon", 2023); | ||
var layer = Layer.PRIMARY; | ||
|
||
testWith( | ||
Arrays.asList(getTestPolygon(polygonId, valid())), // | ||
Arrays.asList(layerMap(getTestPrimaryLayer(polygonId, valid()))), // | ||
Arrays.asList(Collections.singletonList(getTestSpecies(polygonId, layer, x -> { | ||
x.setPercentGenus(101f); | ||
}))), // | ||
app -> { | ||
|
||
var ex = assertThrows(ProcessingException.class, () -> app.process()); | ||
|
||
assertThat( | ||
ex, | ||
hasProperty( | ||
"message", | ||
is( | ||
"Polygon " + polygonId | ||
+ " has PRIMARY layer where species entries have a percentage total that does not sum to 100%." | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. percentage total > 100% |
||
) | ||
) | ||
); | ||
} | ||
); | ||
|
||
} | ||
|
||
@Test | ||
public void testTwoSpeciesSumTo100Percent() throws Exception { | ||
|
||
var polygonId = polygonId("Test Polygon", 2023); | ||
var layer = Layer.PRIMARY; | ||
|
||
testWith( | ||
Arrays.asList(getTestPolygon(polygonId, valid())), // | ||
Arrays.asList(layerMap(getTestPrimaryLayer(polygonId, valid()))), // | ||
Arrays.asList( | ||
Arrays.asList( | ||
// | ||
getTestSpecies(polygonId, layer, "B", x -> { | ||
x.setPercentGenus(75f); | ||
}), getTestSpecies(polygonId, layer, "P", x -> { | ||
x.setPercentGenus(25f); | ||
}) | ||
) | ||
), // | ||
app -> { | ||
|
||
assertDoesNotThrow(() -> app.process()); | ||
|
||
} | ||
); | ||
|
||
} | ||
|
||
@Test | ||
public void testTwoSpeciesSumToLessThan100Percent() throws Exception { | ||
|
||
var polygonId = polygonId("Test Polygon", 2023); | ||
var layer = Layer.PRIMARY; | ||
|
||
testWith( | ||
Arrays.asList(getTestPolygon(polygonId, valid())), // | ||
Arrays.asList(layerMap(getTestPrimaryLayer(polygonId, valid()))), // | ||
Arrays.asList( | ||
Arrays.asList( | ||
// | ||
getTestSpecies(polygonId, layer, "B", x -> { | ||
x.setPercentGenus(75 - 1f); | ||
}), getTestSpecies(polygonId, layer, "P", x -> { | ||
x.setPercentGenus(25f); | ||
}) | ||
) | ||
), // | ||
app -> { | ||
|
||
var ex = assertThrows(ProcessingException.class, () -> app.process()); | ||
|
||
assertThat( | ||
ex, | ||
hasProperty( | ||
"message", | ||
is( | ||
"Polygon " + polygonId | ||
+ " has PRIMARY layer where species entries have a percentage total that does not sum to 100%." | ||
) | ||
) | ||
); | ||
} | ||
); | ||
|
||
} | ||
|
||
@Test | ||
public void testTwoSpeciesSumToMoreThan100Percent() throws Exception { | ||
|
||
var polygonId = polygonId("Test Polygon", 2023); | ||
var layer = Layer.PRIMARY; | ||
|
||
testWith( | ||
Arrays.asList(getTestPolygon(polygonId, valid())), // | ||
Arrays.asList(layerMap(getTestPrimaryLayer(polygonId, valid()))), // | ||
Arrays.asList( | ||
Arrays.asList( | ||
// | ||
getTestSpecies(polygonId, layer, "B", x -> { | ||
x.setPercentGenus(75 + 1f); | ||
}), getTestSpecies(polygonId, layer, "P", x -> { | ||
x.setPercentGenus(25f); | ||
}) | ||
) | ||
), // | ||
app -> { | ||
|
||
var ex = assertThrows(ProcessingException.class, () -> app.process()); | ||
|
||
assertThat( | ||
ex, | ||
hasProperty( | ||
"message", | ||
is( | ||
"Polygon " + polygonId | ||
+ " has PRIMARY layer where species entries have a percentage total that does not sum to 100%." | ||
) | ||
) | ||
); | ||
} | ||
); | ||
|
||
} | ||
|
||
private static <T> MockStreamingParser<T> | ||
mockStream(IMocksControl control, Map<String, Object> controlMap, String key, String name) | ||
throws IOException { | ||
|
@@ -474,10 +640,14 @@ FipLayer getTestVeteranLayer(String polygonId, Consumer<FipLayer> mutator) { | |
}; | ||
|
||
FipSpecies getTestSpecies(String polygonId, Layer layer, Consumer<FipSpecies> mutator) { | ||
return getTestSpecies(polygonId, layer, "B", mutator); | ||
}; | ||
|
||
FipSpecies getTestSpecies(String polygonId, Layer layer, String genusId, Consumer<FipSpecies> mutator) { | ||
var result = new FipSpecies( | ||
polygonId, // polygonIdentifier | ||
layer, // layer | ||
"B", // genus | ||
genusId, // genus | ||
100.0f, // percentGenus | ||
Collections.emptyMap() // speciesPercent | ||
); | ||
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In theory, this test is actually testing that the percentage is greater than 100 and thus the Validation Error should state that the sum > 100%.