Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Key Field is incorrect for GROUP BY complex expressions #2740

Closed
big-andy-coates opened this issue Apr 28, 2019 · 2 comments
Closed

Key Field is incorrect for GROUP BY complex expressions #2740

big-andy-coates opened this issue Apr 28, 2019 · 2 comments

Comments

@big-andy-coates
Copy link
Contributor

Given a C*AS statement that contains a GROUP BY clause that is anything more than a simple column reference AND where a matching expression can be found in the projection THEN the key field of the resultant source stored in the metastore is incorrect.

The net result of this is we miss an opportunity to avoid a repartition step on downstream queries that require the source to be (re-)keyed on the field.

e.g.

Given the SQL statement:

CREATE TABLE OUTPUT AS SELECT foo + bar, COUNT(*) FROM INPUT GROUP BY foo + bar;

Expected result: The keyField of OUTPUT should be set to KSQL_COL_0, i.e. the name of the first column in the projection, as this matches the GROUP BY clause.

Actual result: the keyField of OUTPUT is set to null.

The same is true with aliased projection fields too:

Given the SQL statement:

CREATE TABLE OUTPUT AS SELECT CAST(ID AS STRING) AS PRODUCT_ID, COUNT(*) FROM INPUT GROUP BY CAST(ID AS STRING);

Expected result: The keyField of OUTPUT should be set to PRODUCT_ID, i.e. the name of the first column in the projection, as this matches the GROUP BY clause.

Actual result: the keyField of OUTPUT is set to null.

@agavra
Copy link
Contributor

agavra commented Dec 5, 2019

@big-andy-coates are we sure we want this behavior? I think with #3982 this is no longer desired - they are two different fields (one key field, one is a value field).

@big-andy-coates
Copy link
Contributor Author

Closing as we've removing WITH(KEY) anyway. #3537

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants