Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: allow reserved keywords on variables names #6572

Merged
merged 1 commit into from
Nov 16, 2020

Conversation

spena
Copy link
Member

@spena spena commented Nov 4, 2020

Description

Issue: The DEFINE/UNDEFINE statements do not allow variables names that matches the symbolic names found in SqlBase.g4. For instance define topic = 't1'; fails because topic is a reserved keyword. However, these should not be a problem for variables names. We should allow any variable name.

The SqlBase.g4 has a rule for nonReservedKeyword which I used initially, but this is not scalable. There are several missing keywords in that rule, and not all statements allow the same keywords as non-reserved. To scale this rule, I added an extra validation to the parser error listener to check if a syntax error is caused by a keyword being used as variable name. If so, then it returns silently without throwing any error, thus allowing the define and undefine statements to continue.

i.e.

# BEFORE
ksql> define topic = 'a';
line 1:8: mismatched input 'topic' expecting {'EMIT', 'CHANGES', 'FINAL', 'ESCAPE', 'INTEGER', 'DATE', 'TIME', 'TIMESTAMP', 'INTERVAL', 'YEAR', 'MONTH', 'DAY', 'HOUR', 'MINUTE', 'SECOND', 'ZONE', 'PARTITION', 'STRUCT', 'EXPLAIN', 'ANALYZE', 'TYPE', 'TYPES', 'SHOW', 'TABLES', 'COLUMNS', 'COLUMN', 'PARTITIONS', 'FUNCTIONS', 'FUNCTION', 'ARRAY', 'MAP', 'SET', 'RESET', 'SESSION', 'KEY', 'SINK', 'SOURCE', 'PRIMARY', 'REPLACE', 'ASSERT', 'ADD', 'ALTER', 'IF', IDENTIFIER}
Statement: define topic = 'a';
Caused by: line 1:8: mismatched input 'topic' expecting {'EMIT', 'CHANGES',
	'FINAL', 'ESCAPE', 'INTEGER', 'DATE', 'TIME', 'TIMESTAMP', 'INTERVAL', 'YEAR',
	'MONTH', 'DAY', 'HOUR', 'MINUTE', 'SECOND', 'ZONE', 'PARTITION', 'STRUCT',
	'EXPLAIN', 'ANALYZE', 'TYPE', 'TYPES', 'SHOW', 'TABLES', 'COLUMNS', 'COLUMN',
	'PARTITIONS', 'FUNCTIONS', 'FUNCTION', 'ARRAY', 'MAP', 'SET', 'RESET',
	'SESSION', 'KEY', 'SINK', 'SOURCE', 'PRIMARY', 'REPLACE', 'ASSERT', 'ADD',
	'ALTER', 'IF', IDENTIFIER}
Caused by: org.antlr.v4.runtime.InputMismatchException

# AFTER 
ksql> define topic = 'a';

ksql> show variables;

 Variable Name | Value 
-----------------------
 topic         | a     
-----------------------

ksql> undefine topic;

This fix allowed me to removed the undesired keywords from the error message.

sql> define 1 = 'a';
line 1:8: mismatched input '1' expecting IDENTIFIER
Statement: define 1 = 'a';
Caused by: line 1:8: mismatched input '1' expecting IDENTIFIER
Caused by: org.antlr.v4.runtime.InputMismatchException

The validator is on SyntaxErrorValidator, which is used as an error listener for the parsers. I just copied the syntaxError implementation from DefaultKsqlParser to this class. We can add more validators in the future if we find another error that should be valid and is difficult to add as rule in SqlBase.g4. We can also add validators to format the error messages for a better user experience.

Testing done

Added unit tests
Verified manually

Reviewer checklist

  • Ensure docs are updated if necessary. (eg. if a user visible feature is being added or changed).
  • Ensure relevant issues are linked (description should include text like "Fixes #")

@spena spena added the bug label Nov 4, 2020
@spena spena requested a review from a team November 4, 2020 15:33
@spena spena force-pushed the fix_vars_keywords branch from 987ff4f to 8a34f7d Compare November 10, 2020 18:39
Copy link
Contributor

@agavra agavra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@spena spena merged commit 2da360a into confluentinc:master Nov 16, 2020
@spena spena deleted the fix_vars_keywords branch November 16, 2020 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants