Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(explore): Druid data type conversion #13595

Closed

Conversation

nikolagigic
Copy link
Contributor

SUMMARY

Data type conversion for Apache Druid engine spec. This is an addition to the base implementation of data type conversion project.
#13294

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TEST PLAN

ADDITIONAL INFORMATION

  • Has associated issue:
  • Changes UI
  • Requires DB Migration.
  • Confirm DB Migration upgrade and downgrade tested.
  • Introduces new feature or API
  • Removes existing feature or API

@nikolagigic nikolagigic changed the title add __time column type mapping feat(explore): Druid data type conversion Mar 12, 2021
@codecov
Copy link

codecov bot commented Mar 12, 2021

Codecov Report

Merging #13595 (6561fdf) into master (609c359) will decrease coverage by 0.12%.
The diff coverage is 74.67%.

❗ Current head 6561fdf differs from pull request most recent head 2bf41c9. Consider uploading reports for the commit 2bf41c9 to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##           master   #13595      +/-   ##
==========================================
- Coverage   77.29%   77.16%   -0.13%     
==========================================
  Files         910      933      +23     
  Lines       46359    47386    +1027     
  Branches     5633     5928     +295     
==========================================
+ Hits        35831    36566     +735     
- Misses      10392    10672     +280     
- Partials      136      148      +12     
Flag Coverage Δ
cypress 56.23% <44.68%> (-0.50%) ⬇️
hive 80.26% <91.64%> (+0.19%) ⬆️
mysql 80.56% <91.86%> (+0.19%) ⬆️
postgres 80.60% <91.86%> (+0.17%) ⬆️
presto ?
python 80.98% <91.86%> (+0.04%) ⬆️
sqlite 80.15% <88.22%> (+0.11%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...et-frontend/src/SqlLab/components/TableElement.jsx 88.37% <ø> (ø)
superset-frontend/src/chart/chartReducer.ts 48.59% <ø> (-15.79%) ⬇️
...ontend/src/common/hooks/usePrevious/usePrevious.ts 100.00% <ø> (ø)
...ontend/src/components/CertifiedIconWithTooltip.tsx 54.54% <0.00%> (-5.46%) ⬇️
...rset-frontend/src/components/ErrorMessage/types.ts 100.00% <ø> (ø)
...nd/src/components/SupersetResourceSelect/index.tsx 90.32% <ø> (ø)
...et-frontend/src/dashboard/actions/nativeFilters.ts 60.00% <ø> (-4.11%) ⬇️
...end/src/dashboard/components/StickyVerticalBar.tsx 100.00% <ø> (ø)
...end/src/dashboard/components/dnd/DragDroppable.jsx 97.29% <ø> (ø)
...c/dashboard/components/gridComponents/Markdown.jsx 90.90% <ø> (+0.52%) ⬆️
... and 217 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 609c359...2bf41c9. Read the comment docs.

@@ -38,6 +52,14 @@ class DruidEngineSpec(BaseEngineSpec): # pylint: disable=abstract-method
allows_joins = False
allows_subqueries = True

column_type_mappings = (
(
re.compile(r"^__time", re.IGNORECASE),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's actually the column name that is __type, not the type. So this needs to be picked up in get_column_spec, probably by adding the column name to the signature.

@pull-request-size pull-request-size bot added size/S and removed size/M labels Mar 12, 2021
@pull-request-size pull-request-size bot added size/M and removed size/S labels Mar 12, 2021
@junlincc junlincc added the explore:datapanel Related to the Data panel of Explore label Mar 14, 2021
Comment on lines +60 to +61
col = TableColumn(column_name="__time", type=column_spec.sqla_type, table=tbl)
self.assertTrue(col)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the purpose of this assertion? Also, the type of the type field on BaseColumn and the child class TableColumn is String(32), i.e. should not be passed a sqla column type.

nikolagigic and others added 2 commits March 23, 2021 16:49
@stale
Copy link

stale bot commented Jun 11, 2021

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.

@stale stale bot added the inactive Inactive for >= 30 days label Jun 11, 2021
@villebro
Copy link
Member

Closing this as new work in this area has made this PR redundant.

@villebro villebro closed this Nov 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
explore:datapanel Related to the Data panel of Explore inactive Inactive for >= 30 days size/M
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants