Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

document arrayContainsElement filter #15455

Merged
merged 4 commits into from
Dec 7, 2023
Merged

document arrayContainsElement filter #15455

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
a8005f6
document arrayContainsElement filter
clintropolis Nov 29, 2023
740f7fb
Merge remote-tracking branch 'upstream/master' into array-contains-el…
clintropolis Dec 7, 2023
21a50e2
review stuff
clintropolis Dec 7, 2023
d8f46ca
more example more better
clintropolis Dec 7, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions docs/querying/filters.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -450,6 +450,43 @@ Note that it is often more optimal to use a like filter instead of a regex for s
{ "type": "regex", "dimension": "someColumn", "pattern": ^50.* }
```

## Array contains element filter

The `arrayContainsElement` filter is for checking if an `ARRAY` contains a specific element, but can match against any type of column with scalar columns being treated as single element arrays.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The `arrayContainsElement` filter is for checking if an `ARRAY` contains a specific element, but can match against any type of column with scalar columns being treated as single element arrays.
The `arrayContainsElement` filter is for checking if an `ARRAY` contains a specific element, but can also match against any type of column. When matching against scalar columns, scalar columns are treated as single-element arrays.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Includes Abhishek's suggestions:

Suggested change
The `arrayContainsElement` filter is for checking if an `ARRAY` contains a specific element, but can match against any type of column with scalar columns being treated as single element arrays.
The `arrayContainsElement` filter checks if an `ARRAY` contains a specific element but can also match against any type of column. When matching against scalar columns, scalar columns are treated as single-element arrays.


| Property | Description | Required |
| -------- | ----------- | -------- |
| `type` | Must be "arrayContainsElement".| Yes |
| `column` | Input column or virtual column name to filter. | Yes |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| `column` | Input column or virtual column name to filter. | Yes |
| `column` | Input column or virtual column name to filter on. | Yes |

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this matches all of the other tables in the page.. i guess i can update all of the tables... again :p

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its ok even if we make an exception here :)

| `elementMatchValueType` | String specifying the type of element value to match. For example `STRING`, `LONG`, `DOUBLE`, `FLOAT`, `ARRAY<STRING>`, `ARRAY<LONG>`, or any other Druid type. The `elementMatchValueType` determines how Druid interprets the `elementMatchValue` to assist in converting to the type of elements contained in the matched `column`. | Yes |
| `elementMatchValue` | Array element value to match, including null. | Yes |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| `elementMatchValue` | Array element value to match, including null. | Yes |
| `elementMatchValue` | The value to is matched against the elements of `column`. This value can be null. | Yes |

my phrasing might not be best here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original wording is mostly ok imo, but the explicit can be null is helpful.

Suggested change
| `elementMatchValue` | Array element value to match, including null. | Yes |
| `elementMatchValue` | Array element value to match. This value can be null. | Yes |


### Example: equivalent of `WHERE ARRAY_CONTAINS(someArrayColumn, 'hello')`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel a language like below is better

Suggested change
### Example: equivalent of `WHERE ARRAY_CONTAINS(someArrayColumn, 'hello')`
### Example: to filter rows where someArrayColumn has any element that is equal to 'hello'

Copy link
Member Author

@clintropolis clintropolis Nov 30, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all of the other examples on this page are framed in terms of equivalent SQL queries, am going to leave it as is

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hehe. that's what I thought but was too lazy to look beyond the surrounding functions.


```json
{ "type": "arrayContainsElement", "column": "someArrayColumn", "elementMatchValueType": "STRING", "elementMatchValue": "hello" }
```

### Example: equivalent of `WHERE ARRAY_CONTAINS(someNumericArrayColumn, 1.23)`

```json
{ "type": "arrayContainsElement", "column": "someNumericArrayColumn", "elementMatchValueType": "DOUBLE", "elementMatchValue": 1.23 }
```

### Example: equivalent of `WHERE ARRAY_CONTAINS(someNumericArrayColumn, ARRAY[1, 2, 3])`

```json
{
"type": "and",
"fields": [
{ "type": "arrayContainsElement", "column": "someNumericArrayColumn", "elementMatchValueType": "LONG", "elementMatchValue": 1 },
{ "type": "arrayContainsElement", "column": "someNumericArrayColumn", "elementMatchValueType": "LONG", "elementMatchValue": 2 },
{ "type": "arrayContainsElement", "column": "someNumericArrayColumn", "elementMatchValueType": "LONG", "elementMatchValue": 3 }
]
}

```

## Interval filter

The Interval filter enables range filtering on columns that contain long millisecond values, with the boundaries specified as ISO 8601 time intervals. It is suitable for the `__time` column, long metric columns, and dimensions with values that can be parsed as long milliseconds.
Expand Down
1 change: 1 addition & 0 deletions website/.spelling
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -246,6 +246,7 @@ aggregators
ambari
analytics
arrayElement
arrayContainsElement
assumeNewlineDelimited
assumeRoleArn
assumeRoleExternalId
Expand Down