Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add nested field support to KQL #47070

Merged
merged 78 commits into from
Oct 30, 2019
Merged
Show file tree
Hide file tree
Changes from 58 commits
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
497a4f9
Add nested sub type
Bargs Aug 30, 2019
52dde94
Broken commit. Starting work on nested kuery function. Still needs work
Bargs Aug 30, 2019
dbd53e0
Merge remote-tracking branch 'upstream/master' into nested
Bargs Sep 5, 2019
ec7046a
Support nested `{}` syntax in KQL
Bargs Sep 5, 2019
cef9d4c
Basic single level nested queries working
Bargs Sep 5, 2019
db95789
Merge remote-tracking branch 'upstream/master' into nested
Bargs Sep 6, 2019
96ad9a8
Merge remote-tracking branch 'upstream/master' into nested
Bargs Sep 9, 2019
7294ad5
support fields that are both nested and multi. And support nested fields
Bargs Sep 10, 2019
b9056f5
Require path when querying nested fields
Bargs Sep 12, 2019
7de3045
Merge remote-tracking branch 'upstream/master' into nested
Bargs Sep 12, 2019
d95ca51
autocomplete work
Bargs Sep 12, 2019
7ed7074
Merge remote-tracking branch 'upstream/master' into nested
Bargs Sep 13, 2019
02c5e23
support nested fields in the autocomplete value suggestions
Bargs Sep 13, 2019
af7723c
fix undefined property access error
Bargs Sep 13, 2019
1708560
Support operator suggestions inside nested groups
Bargs Sep 13, 2019
2a7ab3a
Merge remote-tracking branch 'upstream/master' into nested
Bargs Sep 19, 2019
443df93
Improve placement of cursor after selecting autocomplete suggestions
Bargs Sep 20, 2019
2c5ea5a
Merge remote-tracking branch 'upstream/master' into nested
Bargs Sep 26, 2019
ff67683
Merge remote-tracking branch 'upstream/master' into nested
Bargs Sep 27, 2019
db4ad9e
Provide helpful error messages when nested syntax is not used correctly
Bargs Sep 27, 2019
376b797
only suggest actual nested fields inside a nested group. Previously this
Bargs Sep 27, 2019
9e2355e
Merge remote-tracking branch 'upstream/master' into nested
Bargs Sep 30, 2019
4154422
Implement nested field syntax info toast
Bargs Oct 1, 2019
80f7ac6
remove comment
Bargs Oct 1, 2019
9cbe1fd
Merge remote-tracking branch 'upstream/master' into nested
Bargs Oct 1, 2019
59fdebe
Merge remote-tracking branch 'upstream/master' into nested
Bargs Oct 4, 2019
dc7a42d
fix outdated reference
Bargs Oct 4, 2019
6f398c4
update expected subType format everywhere
Bargs Oct 4, 2019
81b5466
Add subType and parent migration for index pattern fields
Bargs Oct 4, 2019
56792d1
Merge remote-tracking branch 'upstream/master' into nested
Bargs Oct 18, 2019
701eb9f
Fixes from merge conflicts
Bargs Oct 18, 2019
28da930
update fixtures with new subType shape
Bargs Oct 18, 2019
23307c9
Merge remote-tracking branch 'upstream/master' into nested
Bargs Oct 22, 2019
8329f76
Update usage of old parent property
Bargs Oct 22, 2019
a0eecb0
Merge remote-tracking branch 'upstream/master' into nested
Bargs Oct 22, 2019
b82982b
Update tests that were written for first syntax I tried
Bargs Oct 22, 2019
2740fc1
Fix geo_polygon tests
Bargs Oct 22, 2019
25ab17d
update test
Bargs Oct 22, 2019
f134405
Harden migration, fixes spaces api integration test failures
Bargs Oct 22, 2019
abb0e84
Merge remote-tracking branch 'upstream/master' into nested
Bargs Oct 22, 2019
2b8b4ba
comment out for now so other tests can run
Bargs Oct 22, 2019
dd3778c
update field_caps_response tests with nested field cases
Bargs Oct 23, 2019
b70c607
add tests for getFullFieldNameNode function
Bargs Oct 23, 2019
75b4007
tests for the handling of nested field suggestions in the autocomplete
Bargs Oct 23, 2019
52bace2
Fix so nested subfields are suggested anytime they contain the search…
Bargs Oct 23, 2019
6df91e2
Merge remote-tracking branch 'upstream/master' into nested
Bargs Oct 23, 2019
801b06c
add doubly nested ast test
Bargs Oct 23, 2019
c60b2ba
add tests for passing nested path to operator and value suggestion pr…
Bargs Oct 23, 2019
ad3d6f3
nested function tests
Bargs Oct 23, 2019
0b52afc
Add tests for nested functionality in existing functions
Bargs Oct 23, 2019
d4f8ae7
Merge remote-tracking branch 'upstream/master' into nested
Bargs Oct 23, 2019
6067712
Fix failures caused by buildNodeParams change
Bargs Oct 24, 2019
adfe6d1
Merge remote-tracking branch 'upstream/master' into nested
Bargs Oct 24, 2019
163b4b9
Tests for nested auto wrapping with wildcard field names
Bargs Oct 24, 2019
e047e82
Add API integration test for nested field support in suggestions API
Bargs Oct 24, 2019
e327f67
Merge remote-tracking branch 'upstream/master' into nested
Bargs Oct 24, 2019
9305b18
Fix incorrect type
Bargs Oct 24, 2019
ecf3994
Add simple functional test to ensure nested queries work end to end
Bargs Oct 24, 2019
ce1f3ef
update tests for fixture changes
Bargs Oct 24, 2019
88790a7
Merge remote-tracking branch 'upstream/master' into nested
Bargs Oct 24, 2019
736d994
Merge remote-tracking branch 'upstream/master' into nested
Bargs Oct 25, 2019
4797092
allow for whitespace in the nested operator
Bargs Oct 25, 2019
e791515
use class member
Bargs Oct 25, 2019
0a5bbf4
Fix bug preventing suggestions from working on index patterns with ch…
Bargs Oct 25, 2019
3a715c2
Merge remote-tracking branch 'upstream/master' into nested
Bargs Oct 25, 2019
9b3c63b
`:{` is no longer expected now that whitespace is allowed between the…
Bargs Oct 25, 2019
9324965
Fix test with updated error message
Bargs Oct 25, 2019
e4dee06
Update error messages
Bargs Oct 25, 2019
6285bf0
Update error message based on copy feedback
Bargs Oct 25, 2019
3bb3947
add nested query syntax info to the KQL documentation
Bargs Oct 25, 2019
4b589a5
Merge remote-tracking branch 'upstream/master' into nested
Bargs Oct 28, 2019
74593c5
Apply suggestions from code review
Bargs Oct 28, 2019
2ce8079
Improve help message copy
Bargs Oct 28, 2019
d34ccfd
Merge remote-tracking branch 'upstream/master' into nested
Bargs Oct 28, 2019
e00128a
Update docs/discover/kuery.asciidoc
Bargs Oct 29, 2019
898f76a
make it clear parts of the query can match different docs, but they d…
Bargs Oct 29, 2019
39ddcf8
Merge remote-tracking branch 'upstream/master' into nested
Bargs Oct 30, 2019
3db6b18
updated outdated service name
Bargs Oct 30, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 23 additions & 2 deletions packages/kbn-es-query/src/__fixtures__/index_pattern_response.json
Original file line number Diff line number Diff line change
Expand Up @@ -161,8 +161,7 @@
"searchable": true,
"aggregatable": true,
"readFromDocValues": true,
"parent": "machine.os",
"subType": "multi"
"subType": { "multi": { "parent": "machine.os" } }
},
{
"name": "geo.src",
Expand Down Expand Up @@ -277,6 +276,28 @@
"searchable": true,
"aggregatable": true,
"readFromDocValues": false
},
{
"name": "nestedField.child",
"type": "string",
"esTypes": ["text"],
"count": 0,
"scripted": false,
"searchable": true,
"aggregatable": false,
"readFromDocValues": false,
"subType": { "nested": { "path": "nestedField" } }
},
{
"name": "nestedField.nestedChild.doublyNestedChild",
"type": "string",
"esTypes": ["text"],
"count": 0,
"scripted": false,
"searchable": true,
"aggregatable": false,
"readFromDocValues": false,
"subType": { "nested": { "path": "nestedField.nestedChild" } }
}
]
}
62 changes: 62 additions & 0 deletions packages/kbn-es-query/src/kuery/ast/__tests__/ast.js
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,68 @@ describe('kuery AST API', function () {
expect(actual).to.eql(expected);
});

it('should support nested queries indicated by curly braces', () => {
const expected = nodeTypes.function.buildNode(
'nested',
'nestedField',
nodeTypes.function.buildNode('is', 'childOfNested', 'foo')
);
const actual = ast.fromKueryExpression('nestedField:{ childOfNested: foo }');
expect(actual).to.eql(expected);
});

it('should support nested subqueries and subqueries inside nested queries', () => {
const expected = nodeTypes.function.buildNode(
'and',
[
nodeTypes.function.buildNode('is', 'response', '200'),
nodeTypes.function.buildNode(
'nested',
'nestedField',
nodeTypes.function.buildNode('or', [
nodeTypes.function.buildNode('is', 'childOfNested', 'foo'),
nodeTypes.function.buildNode('is', 'childOfNested', 'bar'),
])
)]);
const actual = ast.fromKueryExpression('response:200 and nestedField:{ childOfNested:foo or childOfNested:bar }');
expect(actual).to.eql(expected);
});

it('should support nested sub-queries inside paren groups', () => {
const expected = nodeTypes.function.buildNode(
'and',
[
nodeTypes.function.buildNode('is', 'response', '200'),
nodeTypes.function.buildNode('or', [
nodeTypes.function.buildNode(
'nested',
'nestedField',
nodeTypes.function.buildNode('is', 'childOfNested', 'foo')
),
nodeTypes.function.buildNode(
'nested',
'nestedField',
nodeTypes.function.buildNode('is', 'childOfNested', 'bar')
),
])
]);
const actual = ast.fromKueryExpression('response:200 and ( nestedField:{ childOfNested:foo } or nestedField:{ childOfNested:bar } )');
expect(actual).to.eql(expected);
});

it('should support nested groups inside other nested groups', () => {
const expected = nodeTypes.function.buildNode(
'nested',
'nestedField',
nodeTypes.function.buildNode(
'nested',
'nestedChild',
nodeTypes.function.buildNode('is', 'doublyNestedChild', 'foo')
)
);
const actual = ast.fromKueryExpression('nestedField:{ nestedChild:{ doublyNestedChild:foo } }');
expect(actual).to.eql(expected);
});
});

describe('fromLiteralExpression', function () {
Expand Down
4 changes: 2 additions & 2 deletions packages/kbn-es-query/src/kuery/ast/ast.js
Original file line number Diff line number Diff line change
Expand Up @@ -63,12 +63,12 @@ function fromExpression(expression, parseOptions = {}, parse = parseKuery) {
* IndexPattern isn't required, but if you pass one in, we can be more intelligent
* about how we craft the queries (e.g. scripted fields)
*/
export function toElasticsearchQuery(node, indexPattern, config = {}) {
export function toElasticsearchQuery(node, indexPattern, config = {}, context = {}) {
if (!node || !node.type || !nodeTypes[node.type]) {
return toElasticsearchQuery(nodeTypes.function.buildNode('and', []));
}

return nodeTypes[node.type].toElasticsearchQuery(node, indexPattern, config);
return nodeTypes[node.type].toElasticsearchQuery(node, indexPattern, config, context);
}

export function doesKueryExpressionHaveLuceneSyntaxError(expression) {
Expand Down
Loading