feat: introduce the sql-based testing tool (YATT) #6051
Conversation
| } | ||
|
|
||
| public enum Type { | ||
| TEST("test"), | ||
| EXPECTED_ERROR("expected.error"), | ||
| EXPECTED_MESSAGE("expected.message"), | ||
| ENABLED("enabled"), |
There was a problem hiding this comment.
removed this because it's not super easy to implement cleanly given the iterative nature of the tests. since the sql file supports multiline comments, it's easy enough to disable a test like this:
/* (disabled)
--@test: foo bar
CREATE STREAM foo...;
INSERT INTO ...;
ASSERT VALUES ...;
*/
| } | ||
| } | ||
|
|
||
| public static void assertStream(final AssertStream assertStatement) { |
There was a problem hiding this comment.
I'll implement these in a future PR since to do it "right" involves some engine refactoring to leverage what we already have in production code
ksqldb-functional-tests/src/main/java/io/confluent/ksql/test/parser/SqlTestLoader.java
Outdated
Show resolved
Hide resolved
| -- this test file is a "meta-test" that tests the basic functionality of the KsqlTester | ||
| -- eventually, we plan to remove this test in favor of migrating the QTTs over to use this format | ||
| -- directly | ||
|
|
There was a problem hiding this comment.
I see anything related to windowing is missing. Is this because it is not supported in this PR yet?
There was a problem hiding this comment.
Windowed keys aren't yet supported in our grammar so it'll take a bit of extra work to add support for windowing - I'll work on this after I have some basic query upgrade tests written
| CREATE STREAM bar AS SELECT * FROM foo; | ||
|
|
||
| INSERT INTO foo (rowtime, id, col1) VALUES (1, 1, 1); | ||
| ASSERT VALUES bar (rowtime, id, col1) VALUES (1, 1, 1); |
There was a problem hiding this comment.
Also, Assert stream and Assert table are missing
There was a problem hiding this comment.
yes, they are not implemented yet (see AssertExecutor)
There was a problem hiding this comment.
I just saw your comment in AssertExecutor so nvm
| * @param path the top-level dir to load | ||
| */ | ||
| public SqlTestLoader(final Predicate<SqlTest> testFilter, final Path path) { | ||
| this.shouldRun = Objects.requireNonNull(testFilter, "testFilter"); |
There was a problem hiding this comment.
How would the shouldRun be used? Will there be tests in a test file that one can indicate that they should be skipped? If so, should we add an example to test.sql?
There was a problem hiding this comment.
the idea was to use shouldRun to filter tests to run or not run based on some regex exp. I wasn't going to implement it in this PR quite yet but I figured I might as well set it up here
Description
First draft at the fully implemented
KsqlTester(see #5965). Review guide:test.sqlto see how the testing tool works end-to-endKsqlTesterTestto see the full test in action. I'm planning on refactoring this further when I make it a standalone tool, but given that I want this to test query upgrades I'm leaving this in a little bit of a rougher state for nowSqlTestLoaderto see how we load testsAssertExecutorto look at asserting mechanism. I'm planning on improving the assertion mechanism to leverage theSELECTformatting of rows when I make this tool public, but for now it just compares the Generic representation of rows directyTesting done
test.sqltests this end to endReviewer checklist