Skip to content

Commit

Permalink
Do not use the NiFi validator to ensure schema file existence
Browse files Browse the repository at this point in the history
We currently use the FILE_EXISTS_VALIDATOR to ensure that the DFDL
Schema File property is set to a file that actually exists. However,
validation errors are not written to logs--they only visible in the NiFi
GUI and are are non-existent in MiNiFi. This can make it difficult to
determine what is wrong if there is a typo in the DFDL Schema File
property.

Instead, this just requires that the DFDL schema property is non-empty
and we try to compile the property value regardless. If the path does
not exist then we follow the normal schema compilation failure path,
which generates an error log message and the flow file routes to the
failure relationship.

Note that this means DFDL schema file existence is not checked until a
flow file is sent to the processor, and a log message is generated for
every flow file with the compilation error diagnostic.
  • Loading branch information
stevedlawrence committed Apr 7, 2023
1 parent 409b2e0 commit af22965
Show file tree
Hide file tree
Showing 2 changed files with 8 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ public abstract class AbstractDaffodilProcessor extends AbstractProcessor {
.description("Full path to the DFDL schema file that is to be used for parsing/unparsing.")
.required(true)
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
.build();

public static final PropertyDescriptor PRE_COMPILED_SCHEMA = new PropertyDescriptor.Builder()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,13 @@ public class TestDaffodilProcessor {
public void testDFDLSchemaNotFound() throws IOException {
final TestRunner testRunner = TestRunners.newTestRunner(DaffodilParse.class);
testRunner.setProperty(DaffodilParse.DFDL_SCHEMA_FILE, "/does/not/exist.dfdl.xsd");
testRunner.assertNotValid();
testRunner.assertValid();
testRunner.enqueue(Paths.get("src/test/resources/TestDaffodilProcessor/tokens.csv"));
testRunner.run();
testRunner.assertAllFlowFilesTransferred(DaffodilParse.REL_FAILURE);
final MockFlowFile original = testRunner.getFlowFilesForRelationship(DaffodilParse.REL_FAILURE).get(0);
final String expectedContent = new String(Files.readAllBytes(Paths.get("src/test/resources/TestDaffodilProcessor/tokens.csv")));
original.assertContentEquals(expectedContent);
}

@Test
Expand Down

0 comments on commit af22965

Please sign in to comment.