[ESQL] date nanos binary comparisons #111908
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
elasticsearchmachine
added
the
Team:Analytics
|
Pinging @elastic/es-analytical-engine (Team:Analytics) |
nik9000
approved these changes
Aug 14, 2024
weizijun
added a commit
to weizijun/elasticsearch
that referenced
this pull request
Aug 16, 2024
* upstream/main: (91 commits) Mute org.elasticsearch.xpack.test.rest.XPackRestIT org.elasticsearch.xpack.test.rest.XPackRestIT elastic#111944 Add audit_unenrolled_* attributes to fleet-agents template (elastic#111909) Fix windows memory locking (elastic#111866) Update OAuth2 OIDC SDK (elastic#108799) Adds a warning about manually mounting snapshots managed by ILM (elastic#111883) Update geoip fixture files and utility methods (elastic#111913) Updated Function Score Query Test with Explain Fixes for 8.15.1 (elastic#111929) Mute org.elasticsearch.xpack.sql.qa.security.JdbcCsvSpecIT org.elasticsearch.xpack.sql.qa.security.JdbcCsvSpecIT elastic#111923 [ESQL] date nanos binary comparisons (elastic#111908) [DOCS] Documents output_field behavior after multiple inference runs (elastic#111875) Add additional BlobCacheMetrics, expose BlobCacheMetrics via SharedBlobCacheService (elastic#111730) Mute org.elasticsearch.xpack.sql.qa.multi_cluster_with_security.JdbcCsvSpecIT org.elasticsearch.xpack.sql.qa.multi_cluster_with_security.JdbcCsvSpecIT elastic#111923 Mute org.elasticsearch.xpack.sql.qa.multi_cluster_with_security.JdbcCsvSpecIT test {agg-ordering.testHistogramDateTimeWithCountAndOrder_2} elastic#111919 Mute org.elasticsearch.xpack.sql.qa.multi_cluster_with_security.JdbcCsvSpecIT test {date.testDateParseHaving} elastic#111921 Mute org.elasticsearch.xpack.sql.qa.multi_cluster_with_security.JdbcCsvSpecIT test {agg-ordering.testHistogramDateTimeWithCountAndOrder_1} elastic#111918 Mute org.elasticsearch.xpack.sql.qa.multi_cluster_with_security.JdbcCsvSpecIT test {datetime.testDateTimeParseHaving} elastic#111922 Mute org.elasticsearch.xpack.sql.qa.single_node.JdbcCsvSpecIT org.elasticsearch.xpack.sql.qa.single_node.JdbcCsvSpecIT elastic#111923 Mute org.elasticsearch.xpack.sql.qa.single_node.JdbcCsvSpecIT test {agg-ordering.testHistogramDateTimeWithCountAndOrder_1} elastic#111918 Mute org.elasticsearch.xpack.sql.qa.single_node.JdbcCsvSpecIT test {datetime.testDateTimeParseHaving} elastic#111922 Mute org.elasticsearch.xpack.sql.qa.single_node.JdbcCsvSpecIT test {date.testDateParseHaving} elastic#111921 ... # Conflicts: # server/src/main/java/org/elasticsearch/TransportVersions.java
cbuescher
pushed a commit
to cbuescher/elasticsearch
that referenced
this pull request
Sep 4, 2024
resolves elastic#109992 Nothing fancy here. Nanosecond dates are still longs, and we can just compare them as longs. Please note that, as mentioned in the linked issue, this only supports comparing date nanos to other date nanos, and not comparing to millisecond dates. With the cast functions added in elastic#111850, users can explicitly cast to millisecond dates (or longs) to compare nanos to other things.
davidkyle
pushed a commit
to davidkyle/elasticsearch
that referenced
this pull request
Sep 5, 2024
resolves elastic#109992 Nothing fancy here. Nanosecond dates are still longs, and we can just compare them as longs. Please note that, as mentioned in the linked issue, this only supports comparing date nanos to other date nanos, and not comparing to millisecond dates. With the cast functions added in elastic#111850, users can explicitly cast to millisecond dates (or longs) to compare nanos to other things.
not-napoleon
added a commit
that referenced
this pull request
Nov 7, 2024
Relates to #109992 When I implemented binary comparisons for date nanos in #111908 I failed to update the verifyBinaryComparisons rule, which implements another layer of type checking for binary comparisons, external to the type logic in the Expression classes. Since the unit tests for the expressions don't run the full query processing stack, this bug was invisible to them. Only full integration tests, such as the CSV tests I've added here, can catch this kind of error. The initial PR did not include these because we didn't have enough supporting functions (notably TO_DATE_NANOS) to write the tests. This PR fixes the issue by adding DATE_NANOS to the list of allowed types for binary comparisons in the verifier.
not-napoleon
added a commit
to not-napoleon/elasticsearch
that referenced
this pull request
Nov 7, 2024
Relates to elastic#109992 When I implemented binary comparisons for date nanos in elastic#111908 I failed to update the verifyBinaryComparisons rule, which implements another layer of type checking for binary comparisons, external to the type logic in the Expression classes. Since the unit tests for the expressions don't run the full query processing stack, this bug was invisible to them. Only full integration tests, such as the CSV tests I've added here, can catch this kind of error. The initial PR did not include these because we didn't have enough supporting functions (notably TO_DATE_NANOS) to write the tests. This PR fixes the issue by adding DATE_NANOS to the list of allowed types for binary comparisons in the verifier.
elasticsearchmachine
pushed a commit
that referenced
this pull request
Nov 7, 2024
Relates to #109992 When I implemented binary comparisons for date nanos in #111908 I failed to update the verifyBinaryComparisons rule, which implements another layer of type checking for binary comparisons, external to the type logic in the Expression classes. Since the unit tests for the expressions don't run the full query processing stack, this bug was invisible to them. Only full integration tests, such as the CSV tests I've added here, can catch this kind of error. The initial PR did not include these because we didn't have enough supporting functions (notably TO_DATE_NANOS) to write the tests. This PR fixes the issue by adding DATE_NANOS to the list of allowed types for binary comparisons in the verifier.
kderusso
pushed a commit
to kderusso/elasticsearch
that referenced
this pull request
Nov 7, 2024
Relates to elastic#109992 When I implemented binary comparisons for date nanos in elastic#111908 I failed to update the verifyBinaryComparisons rule, which implements another layer of type checking for binary comparisons, external to the type logic in the Expression classes. Since the unit tests for the expressions don't run the full query processing stack, this bug was invisible to them. Only full integration tests, such as the CSV tests I've added here, can catch this kind of error. The initial PR did not include these because we didn't have enough supporting functions (notably TO_DATE_NANOS) to write the tests. This PR fixes the issue by adding DATE_NANOS to the list of allowed types for binary comparisons in the verifier.
jozala
pushed a commit
that referenced
this pull request
Nov 13, 2024
Relates to #109992 When I implemented binary comparisons for date nanos in #111908 I failed to update the verifyBinaryComparisons rule, which implements another layer of type checking for binary comparisons, external to the type logic in the Expression classes. Since the unit tests for the expressions don't run the full query processing stack, this bug was invisible to them. Only full integration tests, such as the CSV tests I've added here, can catch this kind of error. The initial PR did not include these because we didn't have enough supporting functions (notably TO_DATE_NANOS) to write the tests. This PR fixes the issue by adding DATE_NANOS to the list of allowed types for binary comparisons in the verifier.
alexey-ivanov-es
pushed a commit
to alexey-ivanov-es/elasticsearch
that referenced
this pull request
Nov 28, 2024
Relates to elastic#109992 When I implemented binary comparisons for date nanos in elastic#111908 I failed to update the verifyBinaryComparisons rule, which implements another layer of type checking for binary comparisons, external to the type logic in the Expression classes. Since the unit tests for the expressions don't run the full query processing stack, this bug was invisible to them. Only full integration tests, such as the CSV tests I've added here, can catch this kind of error. The initial PR did not include these because we didn't have enough supporting functions (notably TO_DATE_NANOS) to write the tests. This PR fixes the issue by adding DATE_NANOS to the list of allowed types for binary comparisons in the verifier.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
resolves #109992
Nothing fancy here. Nanosecond dates are still longs, and we can just compare them as longs. Please note that, as mentioned in the linked issue, this only supports comparing date nanos to other date nanos, and not comparing to millisecond dates. With the cast functions added in #111850, users can explicitly cast to millisecond dates (or longs) to compare nanos to other things.