-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PARTITION BY should support nested fields #2218
Comments
Looks like it is possible to get by with just one stream, as long as you select & alias the nested field first?
It even looks to work with CASE logic, too -
|
Confirmed this works: {
"name": "stream | initially set | partition by struct dereference expression | key in value | no aliasing",
"statements": [
"CREATE STREAM INPUT (foo VARCHAR, bar STRUCT<val VARCHAR>) WITH (kafka_topic='input_topic', key='foo', value_format='JSON');",
"CREATE STREAM OUTPUT AS SELECT bar FROM INPUT PARTITION BY bar->val;"
],
"inputs": [
{"topic": "input_topic", "key": "1", "value": {"foo": "1", "bar": {"val": "val"}}}
],
"outputs": [
{"topic": "OUTPUT", "key": "val", "value": {"BAR": {"VAL": "val"}}}
],
"post": {
"sources": [
{"name": "OUTPUT", "type": "stream", "keyField": null}
]
}
}, I won't commit this because there is already a test for expression support and this doesn't add any coverage. |
Try to repartition stream based on AFTER->ID:
Workaround is a two-stream approach, to flatten the column and then re-partition:
The text was updated successfully, but these errors were encountered: