Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] Fixed-point support for to_arrow and from_arrow #7116

Closed
razajafri opened this issue Jan 11, 2021 · 5 comments · Fixed by #7609
Closed

[FEA] Fixed-point support for to_arrow and from_arrow #7116

razajafri opened this issue Jan 11, 2021 · 5 comments · Fixed by #7609
Assignees
@shwina @codereport
Labels
feature request New feature or request libcudf Affects libcudf (C++/CUDA) code. Spark Functionality that helps Spark RAPIDS

Comments

@razajafri
Copy link
Contributor

Is your feature request related to a problem? Please describe.
Spark supports fixed-point precision for conversion to/from Arrow

Describe the solution you'd like
We need to support to_arrow and from_arrow for fixed-point precision

Additional context
This builds upon #3556
@razajafri razajafri added feature request New feature or request Needs Triage Need team to review and classify Spark Functionality that helps Spark RAPIDS labels Jan 11, 2021
@razajafri razajafri mentioned this issue Jan 11, 2021
Closed
27 tasks
@jrhemstad
Copy link
Contributor

This will be a little tricky since Arrow uses 128bit integers for its fixed-point decimal representation. We'll need some conversion logic and such.

@kkraus14
Copy link
Collaborator

I believe @shwina has worked out the logic in Python in #6715 and ideally we could move that directly to libcudf.

@harrism harrism mentioned this issue Jan 20, 2021
Closed
45 tasks
@kkraus14 kkraus14 added libcudf Affects libcudf (C++/CUDA) code. and removed Needs Triage Need team to review and classify labels Jan 20, 2021
@tgravescs tgravescs mentioned this issue Feb 1, 2021
Open
@github-actions
Copy link

github-actions bot commented Mar 5, 2021

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

@kkraus14
Copy link
Collaborator

kkraus14 commented Mar 5, 2021

This is planned to be tackled in 0.19.

@shwina
Copy link
Contributor

shwina commented Mar 16, 2021

@codereport and I will be pairing on this, PR should be up shortly!

@shwina shwina mentioned this issue Mar 16, 2021
Merged
rapids-bot bot pushed a commit that referenced this issue Apr 12, 2021
@shwina
Add to/from arrow for fixed point columns (#7609)
f593e34 
Closes #7116

Authors:
  - Ashwin Srinath (https://github.com/shwina)
  - Conor Hoekstra (https://github.com/codereport)

Approvers:
  - Keith Kraus (https://github.com/kkraus14)
  - GALI PREM SAGAR (https://github.com/galipremsagar)
  - Vukasin Milovanovic (https://github.com/vuule)
  - Jake Hemstad (https://github.com/jrhemstad)

URL: #7609
devavret pushed a commit to harrism/cudf that referenced this issue Apr 14, 2021
@shwina @devavret
Add to/from arrow for fixed point columns (rapidsai#7609)
e54bffd 
Closes rapidsai#7116

Authors:
  - Ashwin Srinath (https://github.com/shwina)
  - Conor Hoekstra (https://github.com/codereport)

Approvers:
  - Keith Kraus (https://github.com/kkraus14)
  - GALI PREM SAGAR (https://github.com/galipremsagar)
  - Vukasin Milovanovic (https://github.com/vuule)
  - Jake Hemstad (https://github.com/jrhemstad)

URL: rapidsai#7609
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@shwina shwina

@codereport codereport

Labels
feature request New feature or request libcudf Affects libcudf (C++/CUDA) code. Spark Functionality that helps Spark RAPIDS
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add to/from arrow for fixed point columns shwina/cudf
5 participants
@shwina @kkraus14 @razajafri @jrhemstad @codereport