Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support anonymous top level columns, i.e. JSON Array. #1351

Closed
big-andy-coates opened this issue May 29, 2018 · 3 comments · Fixed by #2793 or #2899
Closed

Support anonymous top level columns, i.e. JSON Array. #1351

big-andy-coates opened this issue May 29, 2018 · 3 comments · Fixed by #2793 or #2899
Assignees

Comments

@big-andy-coates
Copy link
Contributor

As per:

https://groups.google.com/forum/#!topic/confluent-platform/XMvBUZEdVTg

If the input topic contains a JSON message that is an array, e.g.

[
   {"nested": 1}
]

We currently don't have a way to access the JSON data.

Hoping this is fixed by #638

@rodesai
Copy link
Contributor

rodesai commented May 31, 2018

This won't get fixed automatically by the struct work (#638). KSQL still needs some way to know how to map the record value to a column. Some options:
- If a schema consists of a single column, and the data in the record is not structured (e.g. an avro record, json object), and the data is compatible with the type in the column, then assign the data in the record to that column
- Add special syntax for specifying that a column maps to the whole record.
- Have an implicit ROWVALUE column that contains the whole recorded value.

@miguno
Copy link
Contributor

miguno commented Jun 27, 2018

@rodesai : Could we address this with a new function similar to EXTRACTJSONFIELD for pulling the data out of the JSON array to get it into a STRUCT, for example?

@almazik
Copy link

almazik commented Sep 25, 2018

What about a single ARRAY<STRUCT<>> field? Not sure if it will be useful though...

big-andy-coates added a commit to big-andy-coates/ksql that referenced this issue May 7, 2019
Also, add support for top-level primitives, maps and arrays. Fixes confluentinc#1351
big-andy-coates added a commit that referenced this issue May 8, 2019
* Switch Serde classes to act on STRUCT not GenericRow.
Also, add support for top-level primitives, maps and arrays. Fixes #1351
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants