Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Serialization of libcudf classes and exposing implementation details #17630

Open
pentschev opened this issue Dec 19, 2024 · 0 comments
Open
Labels
feature request New feature or request

Comments

@pentschev
Copy link
Member

Is your feature request related to a problem? Please describe.
For multi-gpu polars, we will require serializing certain data in Python to be passed between Dask workers, for example aggregations. In #17469 I've proposed a way to do that, however, that proposal requires certain implementation details from aggreation.hpp,
more specifically classes derived from aggregation, such as std_var_aggregation. @vyasr has pointed out to the fact that those details are not exposed to pylibcudf and would be best if it continues like that.

Describe the solution you'd like
The solution proposed in #17469 seems to be the lowest hanging fruit, but as described above may not be considered optimal for several reasons.

Describe alternatives you've considered
Exposing attributes of the classes publicly may be an alternative, but that would incur in a different set of potential issues.

I'm not familiar with most of the design and options available in libcudf, so it's likely core developers will see other potentially better alternatives.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant