serde_arrow icon indicating copy to clipboard operation
serde_arrow copied to clipboard

Use human_readble = false throughout all serializers, deserializers

Open chmp opened this issue 1 year ago • 5 comments

At the moment, the flag is inconsistently set to false in some places. It defaults to true in Serde, but may lead to larger than required files.

What is the effect for commonly used packages?

  • chrono
  • uuid
  • bigdecimal, ...

chmp avatar Aug 03 '24 08:08 chmp

I am now interested in this, I think it is serializing our timestamp (ns since epoch) time as a String because of this...?

sthornington avatar Apr 02 '25 15:04 sthornington

I assume you have some kind of chrono or jiff type somewhere, right?

  • Chrono always serializes its DateTime type as string (source).
  • Jiff as well (source).
  • UUID changes its representation (source)

So the behavior depends a bit on how your timestamps are represented.

chmp avatar Apr 02 '25 20:04 chmp

We have our own which changes how it serializes based on that flag. For now, I have done an .overwrite to TimeStamp(Nanoseconds, Some("UTC")) which seems reasonable...

sthornington avatar Apr 02 '25 21:04 sthornington

This came up in another newtype that we have, it seemed to be getting serialized as a string instead of an integer based on a branch on human_readable in the impl Serialize.

sthornington avatar Apr 05 '25 03:04 sthornington

The inconsistency is not great. However the bigger issue for me here is that I don't know what the default should be.

chmp avatar Apr 12 '25 08:04 chmp