New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Kryo #12

Open

derrickoswald opened this issue Aug 21, 2019 · 0 comments

Labels

Owner

derrickoswald commented Aug 21, 2019

Spark recommends Kryo serialization.
This issue tracks tasks to ensure that this serialization framework is used and works correctly.

find and fix cases where generated RDD are not using Kryo (e.g. MEMORY_ONLY)
implement built-in Kryo serialization to take advantage of the bitfields information that stores which fields are populated
write and perform benchmark tests to check if it's really better

derrickoswald added the enhancement label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment