Upgrade to Kryo 4.0.0 #258
Conversation
|
|
||
| def write(kser: Kryo, out: Output, obj: ClassManifest[T]) { | ||
| kser.writeObject(out, obj.erasure) | ||
| class ClassTagSerializer[T] extends KSerializer[ClassTag[T]] { |
There was a problem hiding this comment.
why remove the old one? This will break people, no?
|
Well, keep ClassManifestSerializer back :) |
|
+1 to Kryo 4.0.0: it seems that #253 is blocked mainly by a dependency on an unreleased version of Kryo, namely 3.1. Upgrading to 4.0.0, will unblock further work on supporting Scala 2.12 (which has just had its first RC published) and help Spark move to 2.12 as well! |
|
+1 |
1 similar comment
|
+1 |
|
@johnynek You're most probably aware of this: in kryo 4 the compatibility issue with the biggest impact IMO is probably caused by |
|
@magro I didn't actually know that. If I make that change as the default setting (which I can do by changing the default scala kryo instantiator here: https://github.com/twitter/chill/blob/develop/chill-scala/src/main/scala/com/twitter/chill/ScalaKryoInstantiator.scala#L56 ), do you expect that serialization of serialized objects will be compatible with 2.21, or just 3.x (were there any changes)? To be honest, after some folks made the mistake of persisting kryo objects, the anxiety about upgrades adds cost to the point that these folks don't even want to try the upgrade. |
|
@johnynek I'd expect that serialization then is compatible with 2.21 if standard |
|
Just wanted to chime in here -- I think we need to discuss if we want to upgrade to kryo 4 just yet. I think we will need kryo3 support at least for quite some time. That or maybe consider shading kryo and removing it form chill's APIs or something like that. But chill is used heavily in scalding / summingbird, and summingbird uses Storm's API, which is only just now using kryo3 as I understand it. |
No description provided.