kryo icon indicating copy to clipboard operation
kryo copied to clipboard

Migration from Kryo 4 to Kryo 5

Open theigl opened this issue 4 years ago • 2 comments

In most cases, Kryo 5 and Kryo 4 are not serialization compatible.

If you need to migrate persistent data, the best approach is to have both Kryo 4 and Kryo 5 on the classpath, read old data with Kryo 4 and write it with Kryo 5.

For more information, see https://github.com/EsotericSoftware/kryo/wiki/Migration-to-v5

theigl avatar Nov 25 '20 08:11 theigl

Might be a good idea to show the change in FieldSerializer configuration. It seems that FieldSerializerConfig needs to be passed into the constructor instead of FieldSerializer itself exposing setter methods to configure after creation. Not sure if that's a general change that should be mentioned?

nicknezis avatar Jan 09 '21 16:01 nicknezis

Hi @theigl , Posting our discussion here.

In the migration guide, you mentioned to use v4 and v5 together.

if (kryo4Version)
  return kryo4.readObject(input4, ArrayList.class);
else
  return kryo5.readObject(input5, ArrayList.class);

Here, how do we determine the value of kryo4Version ? I thought of the following approach and want your opinion on it.

  1. Add a wrapper around kryo4 and kryo5 and add methods for registration, read, write.
  2. Add a custom class resolver for kryo5. a. In writeClass method, add a custom byte with some known value to indicate that this output is written by v5 b. In readClass, read the byte. and call super.readClass()
  3. Add a custom method isVersion5(Input)​ to return true/false based on the byte value.
  4. Before calling kryo.readClassAndObject(), call isVersion5()​ and use that in the above if/else condition

After some time, all of the data will eventually be v5 only. Does this approach seem fine ? Or do you have some better approach to do this migration ?

Our use case doesn't have any persisted data serialized by kryo, its all transient data present in Queues for short term. Also we have more than 100 classes registered with Kryo so we can't modify every class.

@theigl you mentioned in the mail

It really depends. Your idea is one possibility if your data is stored somewhere (e.g. a DB or a queue) without any wrapper. If you have some wrapper around Kryo-serialized data that contains metadata (creationDate etc) it makes sense to put the info there or use the creationDate directly to determine the Kryo version.

We actually don't have wrappers around serialised data.

Are there any edgecases I might be missing by adding an extra byte to the output ?

kapilgarg1996 avatar Feb 06 '24 10:02 kapilgarg1996