Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Render tables partitioned by non-string columns #1533

Merged
merged 10 commits into from
Sep 27, 2023

Conversation

georgecwan
Copy link
Contributor

@georgecwan georgecwan commented Sep 21, 2023

  • Partitioned tables were unable to be rendered if the partitioned column type is not java.lang.String
  • Grid can now render string, numerical, and character columns
  • Fixes partitioning columns filter dropdown to correctly display char types instead of their ASCII values
  • The partition search bar function is now able to search for numbers
  • Removes the handleFocus function for PartitionSelectorSearch since it made it impossible for the user to edit the search bar after deselecting it
  • Fixes Partition-selection drop-downs don't work properly for non-String columns #1441

@georgecwan georgecwan requested a review from mofojed September 21, 2023 20:29
@georgecwan georgecwan self-assigned this Sep 21, 2023
@georgecwan georgecwan marked this pull request as draft September 21, 2023 20:45
@codecov
Copy link

codecov bot commented Sep 22, 2023

Codecov Report

Attention: 18 lines in your changes are missing coverage. Please review.

Comparison is base (d796f83) 46.42% compared to head (aff1bf2) 46.41%.
Report is 7 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1533      +/-   ##
==========================================
- Coverage   46.42%   46.41%   -0.02%     
==========================================
  Files         558      564       +6     
  Lines       35714    35765      +51     
  Branches     8916     8946      +30     
==========================================
+ Hits        16582    16599      +17     
- Misses      19082    19116      +34     
  Partials       50       50              
Flag Coverage Δ
unit 46.41% <45.45%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
packages/iris-grid/src/PartitionSelectorSearch.tsx 69.75% <81.25%> (-4.11%) ⬇️
...ckages/iris-grid/src/IrisGridPartitionSelector.tsx 34.48% <22.22%> (-0.97%) ⬇️
packages/iris-grid/src/IrisGrid.tsx 26.84% <0.00%> (-0.13%) ⬇️

... and 28 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@georgecwan georgecwan marked this pull request as ready for review September 22, 2023 14:15
@georgecwan
Copy link
Contributor Author

There is still an error that occurs when trying to partition by a boolean column:

heduler-Concurrent-2 | i.d.s.s.SessionService    | Internal Error '5c0a20aa-a105-4ad6-b89f-f1aab9995272' java.lang.IllegalArgumentException: Unknown type boolean for MatchFilter value auto-conversion

The code I ran is:

from deephaven import empty_table

part = empty_table(4).update("II=ii")

from deephaven.parquet import write, read

write(part, "/tmp/bool-test/boolCol=true/part.parquet")
write(part, "/tmp/bool-test/boolCol=false/part.parquet")

partition_table = read("/tmp/bool-test")

I suspect that the type of the boolCol being boolean instead of java.lang.Boolean might be related to the issue. I am not sure how to resolve this, some help would be appreciated.

@georgecwan georgecwan force-pushed the 1441-partition-non-string branch from dae7d20 to 51a2246 Compare September 22, 2023 18:51
@mofojed
Copy link
Member

mofojed commented Sep 25, 2023

@georgecwan For the different column type (boolean vs java.lang.Boolean), I'd check in #ddl-team-dhc if that is expected or not. It seems like boolean is always promoted to a java.lang.Boolean when you create a new table, but not in this case. E.g. if you do:

from deephaven import empty_table
t = empty_table(100).update("x=(boolean)(i%2==0)")
print(t.columns)

You get:

[Column(name='x', data_type=java.lang.Boolean, component_type=None, column_type=NORMAL)]

@georgecwan georgecwan changed the title fix: Render table partitioned by non-string columns fix: Render tables partitioned by non-string columns Sep 25, 2023
@georgecwan georgecwan force-pushed the 1441-partition-non-string branch from a10df88 to 1704b6a Compare September 25, 2023 15:12
@georgecwan georgecwan requested a review from mofojed September 25, 2023 15:12
@georgecwan georgecwan force-pushed the 1441-partition-non-string branch from d4b6bd5 to 93be2f5 Compare September 25, 2023 17:41
@georgecwan
Copy link
Contributor Author

georgecwan commented Sep 25, 2023

Error with types boolean vs java.lang.Boolean will be resolved in deephaven/deephaven-core#4547. I have tested my changes against that PR and tables partitioned on boolean columns will work on that version of dh-core.

packages/iris-grid/src/IrisGrid.tsx Outdated Show resolved Hide resolved
packages/iris-grid/src/IrisGrid.tsx Outdated Show resolved Hide resolved
packages/iris-grid/src/IrisGrid.tsx Outdated Show resolved Hide resolved
packages/iris-grid/src/IrisGrid.tsx Outdated Show resolved Hide resolved
packages/iris-grid/src/IrisGridPartitionSelector.tsx Outdated Show resolved Hide resolved
packages/iris-grid/src/PartitionSelectorSearch.tsx Outdated Show resolved Hide resolved
@georgecwan georgecwan requested a review from mofojed September 26, 2023 19:36
@georgecwan georgecwan merged commit 585b2ff into deephaven:main Sep 27, 2023
@georgecwan georgecwan deleted the 1441-partition-non-string branch September 27, 2023 14:20
@github-actions github-actions bot locked and limited conversation to collaborators Sep 27, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Partition-selection drop-downs don't work properly for non-String columns
2 participants