Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EQL: Add match function implementation #55182

Merged
merged 9 commits into from
May 4, 2020
Merged

EQL: Add match function implementation #55182

merged 9 commits into from
May 4, 2020

Conversation

rw-access
Copy link
Contributor

Closes #55178

Discovered that per https://www.elastic.co/guide/en/elasticsearch/reference/current/regexp-syntax.html, character classes aren't supported.

@jrodewig I think this may be worth noting in SQL and EQL docs.

@rw-access rw-access added the :Analytics/EQL EQL querying label Apr 14, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-ql (:Query Languages/EQL)

Copy link
Contributor

@astefan astefan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to see more tests for this function and properly deal with matchLite (see comment review).

@@ -62,53 +62,61 @@ public void testBetweenWrongTypeParams() {
error("process where between(process_name, \"s\", \"e\", false, 2)"));
}

public void testMatchWithText() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No test for regexes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added more in 44af6a6

@@ -62,51 +62,78 @@ public void testBetweenWrongTypeParams() {
error("process where between(process_name, \"s\", \"e\", false, 2)"));
}

public void testCIDRMatchNonIPField() {
public void testCIDRMatchAgainstField() {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I rearranged these test methods alphabetically in hopes that it makes git conflicts less likely

Copy link
Contributor

@astefan astefan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@aleksmaus aleksmaus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

couple of comments, otherwise LGTM

String msg = e.getMessage();
assertEquals("Found 1 problem\n" +
"line 1:15: second argument of [match(process_name, 1)] " +
"must be [string], found value [1] type [integer]", msg);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a test where match is passed only one argument?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about?

process where match(process_name, null)

Copy link
Member

@costin costin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@@ -42,6 +43,7 @@ public EqlFunctionRegistry() {
def(EndsWith.class, EndsWith::new, "endswith"),
def(IndexOf.class, IndexOf::new, "indexof"),
def(Length.class, Length::new, "length"),
def(Match.class, Match::new, "match", "matchlite"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

matchLite as an alias to match?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I originally had "matchLite" but apparently the aliases have to also be normalized to lowercase, so it's "matchlite"

both functions have been around for a while, but matchLite was more limited than regex -- had character clasess and *, *?, and + because of our underlying implementation.

now, they both have the same functionality, so the alias is just for backwards compatibility.

@rw-access rw-access merged commit cc81fe3 into elastic:master May 4, 2020
@rw-access rw-access deleted the eql/match-function branch May 4, 2020 15:28
rw-access added a commit that referenced this pull request May 4, 2020
* EQL: Add Match function
* EQL: Add note about character classes
* EQL: QueryFolderFailTests.java
* EQL: Add match() fail tests
* EQL: Add match tests and fix alias
* EQL: Add match verifier failure tests
* EQL: Reorder query folder fail tests
@rw-access
Copy link
Contributor Author

7.x backport 6da686c

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

EQL: implement match function
5 participants