[lang] Add escape syntax for field names #146
Merged
+40
−1
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
this is an awesome idea and a problem I've run into a bunch personally! Thanks for tackling this |
rcoh
requested changes
Jul 23, 2021
src/lang.rs
Outdated
| start: take_while1!(starts_ident) >> | ||
| rest: take_while!(is_ident) >> | ||
| (start.fragment.0.to_owned() + rest.fragment.0) | ||
| )); | ||
|
|
||
| named!(escaped_ident<Span, String>, do_parse!( |
There was a problem hiding this comment.
what if you use the quoted_string construction instead? that will handle considerably more flexible characters (including escaping quotes inside of the string)
There was a problem hiding this comment.
Good call! I've swapped in quoted_string as you suggested and added a new test case in parse_quoted_ident.
Technically I think JSON and logfmt require double quotes for strings, but I think it's probably better to be permissive since we can receive unstructured input.
Related to rcoh#99 Currently field names containing a space, period, or escaped quote, e.g. `date received` or `grpc.method`, cannot be parsed. This could be worked around using `jq` or similar tools to rewrite the field name, but that's a pain. This commit adds an escaped field name syntax of `["<FIELD>"]`. This is based on the Object Identifier-Index syntax[0] used by `jq`, so it should be somewhat familiar to many people who parse JSON on the command line. The more obvious option of delimiting with just quotes, e.g. "date received", creates an ambiguity between string literals and escaped field names. For example, does `where foo == "date received"` mean field `foo` matches field `date received`, or field `foo` matches the string "date received"? Example query: ``` * | json | where ["grpc.method"] == "Foo" | count by ["date received"] ``` [0] https://stedolan.github.io/jq/manual/#ObjectIdentifier-Index:.foo,.foo.bar
wfchandler
force-pushed
the
escaped_idents
branch
from
79658c7 to
c37a3c6
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related to #99
Currently field names containing a space or period, e.g.
date receivedorgrpc.method, cannot be parsed. This could be worked around usingjqor similar tools to rewrite the field name, but that's a pain.This commit adds an escaped field name syntax of
["<FIELD>"]. This is based on the Object Identifier-Index syntax used byjq, so it should be somewhat familiar to many people who parse JSON on the command line.The more obvious option of delimiting with just quotes, e.g. "date received", creates an ambiguity between string literals and escaped field names. For example, does
where foo == "date received"mean fieldfoomatches fielddate received, or fieldfoomatches the string "date received"?Example query:
agrind '* | json | where ["grpc.method"] == "Foo" | count by ["date received"]'