upickle icon indicating copy to clipboard operation
upickle copied to clipboard

Preparing for uPickle 2.0.0

Open lihaoyi opened this issue 2 years ago • 3 comments

This is the super ticket tracking the release of uPickle 2.0.0. This release is expect to contain the first set of major breaking changes for uPickle after several years

  • [ ] https://github.com/com-lihaoyi/upickle/pull/381
  • [ ] https://github.com/com-lihaoyi/upickle/pull/382
  • [ ] https://github.com/com-lihaoyi/upickle/pull/378

These are breaking changes in the serialization format, which can potentially cause issues to people who have large amounts of serialized data stored: in database JSON columns, S3 blob stores, document databases, etc.. It would also cause issues for people with large systems where upgrading the version of uPickle "atomically" is impossible. We thus need to ensure migrations are possible:

  • [x] Release a new version of uPickle 1.x that is able to read the new serialization format: dictionary-based maps, string-based case objects, etc.. That would allow systems to upgrade piecemeal to the latest version of uPickle 1.x, before upgrading piecemeal to uPickle 2.0.0, without the need for any "big bang" migration.

uPickle 2.0.0 is the chance to get any breaking changes we want into the library, so we'll take it slow to avoid the churn of further breaking changes in the near future. If anyone has other breaking changes that they think should get done, please raise it in this ticket for discussion. Hopefully we can get everything important into before cutting a 2.0.0 release. We'll target the release for 30th April 2022 to give ample time for discussion

lihaoyi avatar Mar 07 '22 01:03 lihaoyi

One breaking change to consider is the discrepancy between:

def visitFloat64StringParts(s: CharSequence, decIndex: Int, expIndex: Int, index: Int): J
def visitFloat64String(s: String, index: Int): J

I haven't investigated whether String or CharSequence is better here, but it might be worth figuring it out and picking one.

On the one hand, CharSequence might let us avoid a String allocation by wrapping a buffer, but on the other, megamorphic calls against CharSequence seem like they could be more expensive. I don't intend to investigate since I'm not going to break the weepickle APIs any time soon, but it's food for thought.

htmldoug avatar Mar 08 '22 21:03 htmldoug

I would love to see changes that allow evolving the library in a binary backward compatible way. For example, converting case classes to normal classes, etc. I didn't check how much effort it is to do such changes though.

lolgab avatar Mar 16 '22 09:03 lolgab

now that 2.0.0 is released shouldn't this be closed?

UnconventionalMindset avatar Jul 24 '22 16:07 UnconventionalMindset