Add The `TO_SECONDS` Function To The SQL Plugin #232

GabeFernandez310 · 2023-02-23T16:21:10Z

Description

Adds the to_seconds function to the SQL plugin. Takes a DATE/TIME/DATETIME/TIMESTAMP/STRING/INTEGER representing a point in time, and converts it to seconds since the year 0. This function is based off of MySQL. For arguments of type TIME, the function takes the current date at the given time. For INTEGER, the integer is parsed as a date (e.g. 950501 == 1995-05-01).

Examples:
SELECT TO_SECONDS(950501); -> 62966505600
SELECT TO_SECONDS('2009-11-29'); -> 63426672000
SELECT TO_SECONDS('2009-11-29 13:43:32'); -> 63426721412

Issues Resolved

#722

Check List

New functionality includes testing.
- All tests pass, including unit test, integration test and doctest
New functionality has been documented.
- New functionality has javadoc added
- New functionality has user manual doc added
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: GabeFernandez310 <[email protected]>

codecov · 2023-02-23T16:29:21Z

Codecov Report

Merging #232 (8d2587e) into integ-add-to_seconds-function (259b001) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@                       Coverage Diff                        @@
##             integ-add-to_seconds-function     #232   +/-   ##
================================================================
  Coverage                            98.38%   98.39%           
- Complexity                            3693     3706   +13     
================================================================
  Files                                  343      343           
  Lines                                 9107     9148   +41     
  Branches                               585      587    +2     
================================================================
+ Hits                                  8960     9001   +41     
  Misses                                 142      142           
  Partials                                 5        5

Flag	Coverage Δ
sql-engine	`98.39% <100.00%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...c/main/java/org/opensearch/sql/expression/DSL.java	`100.00% <100.00%> (ø)`
...arch/sql/expression/datetime/DateTimeFunction.java	`100.00% <100.00%> (ø)`
...h/sql/expression/function/BuiltinFunctionName.java	`100.00% <100.00%> (ø)`
...a/org/opensearch/sql/utils/DateTimeFormatters.java	`100.00% <100.00%> (ø)`

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

core/src/main/java/org/opensearch/sql/expression/datetime/DateTimeFunction.java

core/src/test/java/org/opensearch/sql/expression/datetime/DateTimeFunctionTest.java

docs/user/dql/functions.rst

core/src/test/java/org/opensearch/sql/expression/datetime/DateTimeFunctionTest.java

docs/user/dql/functions.rst

core/src/test/java/org/opensearch/sql/expression/datetime/DateTimeFunctionTest.java

Signed-off-by: GabeFernandez310 <[email protected]>

core/src/main/java/org/opensearch/sql/expression/datetime/DateTimeFunction.java

Signed-off-by: GabeFernandez310 <[email protected]>

core/src/main/java/org/opensearch/sql/expression/datetime/DateTimeFunction.java

core/src/main/java/org/opensearch/sql/utils/DateTimeFormatters.java

core/src/test/java/org/opensearch/sql/expression/datetime/ToSecondsTest.java

GumpacG · 2023-03-02T18:46:16Z

core/src/main/java/org/opensearch/sql/expression/datetime/DateTimeFunction.java

+  private ExprValue exprToSecondsForIntType(ExprValue dateExpr) {
+    try {
+      if (dateExpr.longValue() < 0 || dateExpr.longValue() > 99999999) {
+        throw new DateTimeException("Integer argument was out of range");
+      }
+
+      try {
+        LocalDate date = LocalDate.parse(String.valueOf(dateExpr.longValue()),
+            DATE_FORMATTER_LONG_YEAR);
+
+        return new ExprLongValue(
+            date.toEpochSecond(LocalTime.MIN, ZoneOffset.UTC)
+                + DAYS_0000_TO_1970 * SECONDS_PER_DAY);
+      } catch (DateTimeParseException ignored) {
+        //ignore parse exception and try next format
+      }
+
+      try {
+        LocalDate date = LocalDate.parse(String.valueOf(dateExpr.longValue()),
+            DATE_FORMATTER_SHORT_YEAR);
+
+        return new ExprLongValue(
+            date.toEpochSecond(LocalTime.MIN, ZoneOffset.UTC)
+                + DAYS_0000_TO_1970 * SECONDS_PER_DAY);
+      } catch (DateTimeParseException ignored) {
+        //ignore parse exception and try next format
+      }
+
+      try {
+        LocalDate date = LocalDate.parse(String.valueOf(dateExpr.longValue()),
+            DATE_FORMATTER_SINGLE_DIGIT_YEAR);
+
+        return new ExprLongValue(
+            date.toEpochSecond(LocalTime.MIN, ZoneOffset.UTC)
+                + DAYS_0000_TO_1970 * SECONDS_PER_DAY);
+      } catch (DateTimeParseException ignored) {
+        //ignore parse exception and try next format
+      }
+
+      try {
+        LocalDate date = LocalDate.parse(String.valueOf(dateExpr.longValue()),
+            DATE_FORMATTER_NO_YEAR);
+
+        return new ExprLongValue(
+            date.toEpochSecond(LocalTime.MIN, ZoneOffset.UTC)
+                + DAYS_0000_TO_1970 * SECONDS_PER_DAY);
+      } catch (DateTimeParseException ignored) {
+        //ignore parse exception and try next format
+      }
+
+      LocalDate date = LocalDate.parse(String.valueOf(dateExpr.longValue()),
+          DATE_FORMATTER_SINGLE_DIGIT_MONTH);
+
+      return new ExprLongValue(
+          date.toEpochSecond(LocalTime.MIN, ZoneOffset.UTC)
+              + DAYS_0000_TO_1970 * SECONDS_PER_DAY);
+    } catch (DateTimeException e) {
+      return ExprNullValue.of();
+    }


I think this can be refactored to reduce duplication of try catches

Addressed in ec908ad. Github formatted the diff strangely so I would suggest just looking at the file.

Signed-off-by: GabeFernandez310 <[email protected]>

acarbonetto · 2023-03-08T00:13:58Z

core/src/main/java/org/opensearch/sql/expression/datetime/DateTimeFunction.java

+   * @return is a DateTimeFormatter that can parse the input.
+   */
+  private DateTimeFormatter getFormatter(int dateAsInt) {
+    if (dateAsInt < 0 || dateAsInt > 99999999) {


99999999 is a magic number. Make this a static constant.

nit: that being said, you're trying to length of the number. Why not convert it to a string and get the string length? I think that makes more sense - although less efficient

This is the main reason I did it this way, for efficiency. Actually, using the length of a string might not be too bad though because it enables me to change the multiple if-statements below into a switch/case block. It would also probably be clearer for people who come back to this and read it later... I'll just try to change it.

Fixed in 2b37b0d

acarbonetto · 2023-03-08T00:14:08Z

core/src/main/java/org/opensearch/sql/expression/datetime/DateTimeFunction.java

+    }
+
+    //Check if dateAsInt is at least 6 digits long
+    if (dateAsInt - 99999 > 0) {


99999 is a magic number

Fixed in 2b37b0d

acarbonetto · 2023-03-08T00:14:21Z

core/src/main/java/org/opensearch/sql/expression/datetime/DateTimeFunction.java

+    }
+
+    //Check if dateAsInt is at least 4 digits long
+    if (dateAsInt - 999 > 0) {


999 magic number

Fixed in 2b37b0d

acarbonetto · 2023-03-08T00:14:27Z

core/src/main/java/org/opensearch/sql/expression/datetime/DateTimeFunction.java

+    }
+
+    //Check if dateAsInt is at least 3 digits long
+    if (dateAsInt - 99 > 0) {


magic number

Fixed in 2b37b0d

Signed-off-by: GabeFernandez310 <[email protected]>

core/src/main/java/org/opensearch/sql/utils/DateTimeFormatters.java

Signed-off-by: GabeFernandez310 <[email protected]>

) * Add The `TO_SECONDS` Function To The SQL Plugin (#232) * Added Basic Tests Signed-off-by: GabeFernandez310 <[email protected]> * Added IT Test Signed-off-by: GabeFernandez310 <[email protected]> * Added Implementation Signed-off-by: GabeFernandez310 <[email protected]> * Changed Integration Tests Signed-off-by: GabeFernandez310 <[email protected]> * Added Test For Time Type Signed-off-by: GabeFernandez310 <[email protected]> * Added Documentation Signed-off-by: GabeFernandez310 <[email protected]> * Addressed PR Comments Signed-off-by: GabeFernandez310 <[email protected]> * Fixed Docs and Implementation Signed-off-by: GabeFernandez310 <[email protected]> * Fixed Checkstyle Signed-off-by: GabeFernandez310 <[email protected]> * Changed DateTimeFormatter Priority Signed-off-by: GabeFernandez310 <[email protected]> * Added More Formatters Signed-off-by: GabeFernandez310 <[email protected]> * Updated Docs Signed-off-by: GabeFernandez310 <[email protected]> * Reworked Implementation For Formatters Signed-off-by: GabeFernandez310 <[email protected]> * Cleanup Signed-off-by: GabeFernandez310 <[email protected]> * Added Test Signed-off-by: GabeFernandez310 <[email protected]> * Fixed Implementation And Code Coverage Signed-off-by: GabeFernandez310 <[email protected]> * Changed getFormatter Function Signed-off-by: GabeFernandez310 <[email protected]> * Added Comments Signed-off-by: GabeFernandez310 <[email protected]> --------- Signed-off-by: GabeFernandez310 <[email protected]> * Removed Unneeded Code Signed-off-by: GabeFernandez310 <[email protected]> --------- Signed-off-by: GabeFernandez310 <[email protected]> (cherry picked from commit d38a6ec)

) * Add The `TO_SECONDS` Function To The SQL Plugin (#232) * Added Basic Tests Signed-off-by: GabeFernandez310 <[email protected]> * Added IT Test Signed-off-by: GabeFernandez310 <[email protected]> * Added Implementation Signed-off-by: GabeFernandez310 <[email protected]> * Changed Integration Tests Signed-off-by: GabeFernandez310 <[email protected]> * Added Test For Time Type Signed-off-by: GabeFernandez310 <[email protected]> * Added Documentation Signed-off-by: GabeFernandez310 <[email protected]> * Addressed PR Comments Signed-off-by: GabeFernandez310 <[email protected]> * Fixed Docs and Implementation Signed-off-by: GabeFernandez310 <[email protected]> * Fixed Checkstyle Signed-off-by: GabeFernandez310 <[email protected]> * Changed DateTimeFormatter Priority Signed-off-by: GabeFernandez310 <[email protected]> * Added More Formatters Signed-off-by: GabeFernandez310 <[email protected]> * Updated Docs Signed-off-by: GabeFernandez310 <[email protected]> * Reworked Implementation For Formatters Signed-off-by: GabeFernandez310 <[email protected]> * Cleanup Signed-off-by: GabeFernandez310 <[email protected]> * Added Test Signed-off-by: GabeFernandez310 <[email protected]> * Fixed Implementation And Code Coverage Signed-off-by: GabeFernandez310 <[email protected]> * Changed getFormatter Function Signed-off-by: GabeFernandez310 <[email protected]> * Added Comments Signed-off-by: GabeFernandez310 <[email protected]> --------- Signed-off-by: GabeFernandez310 <[email protected]> * Removed Unneeded Code Signed-off-by: GabeFernandez310 <[email protected]> --------- Signed-off-by: GabeFernandez310 <[email protected]>

) (opensearch-project#1447) * Add The `TO_SECONDS` Function To The SQL Plugin (#232) * Added Basic Tests Signed-off-by: GabeFernandez310 <[email protected]> * Added IT Test Signed-off-by: GabeFernandez310 <[email protected]> * Added Implementation Signed-off-by: GabeFernandez310 <[email protected]> * Changed Integration Tests Signed-off-by: GabeFernandez310 <[email protected]> * Added Test For Time Type Signed-off-by: GabeFernandez310 <[email protected]> * Added Documentation Signed-off-by: GabeFernandez310 <[email protected]> * Addressed PR Comments Signed-off-by: GabeFernandez310 <[email protected]> * Fixed Docs and Implementation Signed-off-by: GabeFernandez310 <[email protected]> * Fixed Checkstyle Signed-off-by: GabeFernandez310 <[email protected]> * Changed DateTimeFormatter Priority Signed-off-by: GabeFernandez310 <[email protected]> * Added More Formatters Signed-off-by: GabeFernandez310 <[email protected]> * Updated Docs Signed-off-by: GabeFernandez310 <[email protected]> * Reworked Implementation For Formatters Signed-off-by: GabeFernandez310 <[email protected]> * Cleanup Signed-off-by: GabeFernandez310 <[email protected]> * Added Test Signed-off-by: GabeFernandez310 <[email protected]> * Fixed Implementation And Code Coverage Signed-off-by: GabeFernandez310 <[email protected]> * Changed getFormatter Function Signed-off-by: GabeFernandez310 <[email protected]> * Added Comments Signed-off-by: GabeFernandez310 <[email protected]> --------- Signed-off-by: GabeFernandez310 <[email protected]> * Removed Unneeded Code Signed-off-by: GabeFernandez310 <[email protected]> --------- Signed-off-by: GabeFernandez310 <[email protected]> (cherry picked from commit d38a6ec)

GabeFernandez310 added 6 commits February 20, 2023 22:02

Added Basic Tests

a94cdef

Signed-off-by: GabeFernandez310 <[email protected]>

Added IT Test

ab66d15

Signed-off-by: GabeFernandez310 <[email protected]>

Added Implementation

3af206e

Signed-off-by: GabeFernandez310 <[email protected]>

Changed Integration Tests

cad2e47

Signed-off-by: GabeFernandez310 <[email protected]>

Added Test For Time Type

51f2410

Signed-off-by: GabeFernandez310 <[email protected]>

Added Documentation

07e0b60

Signed-off-by: GabeFernandez310 <[email protected]>

Yury-Fridlyand requested changes Feb 23, 2023

View reviewed changes

Yury-Fridlyand reviewed Feb 24, 2023

View reviewed changes

core/src/test/java/org/opensearch/sql/expression/datetime/DateTimeFunctionTest.java Outdated Show resolved Hide resolved

Addressed PR Comments

9a2377a

Signed-off-by: GabeFernandez310 <[email protected]>

GabeFernandez310 commented Feb 28, 2023

View reviewed changes

core/src/main/java/org/opensearch/sql/expression/datetime/DateTimeFunction.java Outdated Show resolved Hide resolved

GabeFernandez310 commented Feb 28, 2023

View reviewed changes

core/src/main/java/org/opensearch/sql/expression/datetime/DateTimeFunction.java Outdated Show resolved Hide resolved

GabeFernandez310 added 5 commits February 28, 2023 16:03

Fixed Docs and Implementation

a2045b9

Signed-off-by: GabeFernandez310 <[email protected]>

Fixed Checkstyle

26f4cda

Signed-off-by: GabeFernandez310 <[email protected]>

Changed DateTimeFormatter Priority

0dcd3ce

Signed-off-by: GabeFernandez310 <[email protected]>

Added More Formatters

d4344d3

Signed-off-by: GabeFernandez310 <[email protected]>

Updated Docs

12b981c

Signed-off-by: GabeFernandez310 <[email protected]>

GabeFernandez310 marked this pull request as ready for review March 1, 2023 23:36

Yury-Fridlyand approved these changes Mar 2, 2023

View reviewed changes

GumpacG reviewed Mar 2, 2023

View reviewed changes

GabeFernandez310 added 3 commits March 2, 2023 11:20

Reworked Implementation For Formatters

ec908ad

Signed-off-by: GabeFernandez310 <[email protected]>

Cleanup

8388705

Signed-off-by: GabeFernandez310 <[email protected]>

Added Test

fb3c19b

Signed-off-by: GabeFernandez310 <[email protected]>

matthewryanwells approved these changes Mar 6, 2023

View reviewed changes

GabeFernandez310 added 2 commits March 7, 2023 08:19

Merge branch 'opensearch-project:main' into dev-add-to_seconds-function

c12849e

Fixed Implementation And Code Coverage

d30f749

Signed-off-by: GabeFernandez310 <[email protected]>

acarbonetto approved these changes Mar 8, 2023

View reviewed changes

GumpacG approved these changes Mar 8, 2023

View reviewed changes

Changed getFormatter Function

2b37b0d

Signed-off-by: GabeFernandez310 <[email protected]>

Yury-Fridlyand approved these changes Mar 8, 2023

View reviewed changes

core/src/main/java/org/opensearch/sql/utils/DateTimeFormatters.java Show resolved Hide resolved

Added Comments

8d2587e

Signed-off-by: GabeFernandez310 <[email protected]>

GabeFernandez310 merged commit b366b76 into integ-add-to_seconds-function Mar 9, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add The `TO_SECONDS` Function To The SQL Plugin #232

Add The `TO_SECONDS` Function To The SQL Plugin #232

GabeFernandez310 commented Feb 23, 2023 •

edited

Loading

codecov bot commented Feb 23, 2023 •

edited

Loading

GumpacG Mar 2, 2023

GabeFernandez310 Mar 2, 2023 •

edited

Loading

acarbonetto Mar 8, 2023

acarbonetto Mar 8, 2023

GabeFernandez310 Mar 8, 2023

GabeFernandez310 Mar 8, 2023

acarbonetto Mar 8, 2023

GabeFernandez310 Mar 8, 2023

acarbonetto Mar 8, 2023

GabeFernandez310 Mar 8, 2023

acarbonetto Mar 8, 2023

GabeFernandez310 Mar 8, 2023

Add The TO_SECONDS Function To The SQL Plugin #232

Add The TO_SECONDS Function To The SQL Plugin #232

Conversation

GabeFernandez310 commented Feb 23, 2023 • edited Loading

Description

Issues Resolved

Check List

codecov bot commented Feb 23, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

GabeFernandez310 Mar 2, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Add The `TO_SECONDS` Function To The SQL Plugin #232

Add The `TO_SECONDS` Function To The SQL Plugin #232

GabeFernandez310 commented Feb 23, 2023 •

edited

Loading

codecov bot commented Feb 23, 2023 •

edited

Loading

GabeFernandez310 Mar 2, 2023 •

edited

Loading