quickwit
quickwit copied to clipboard
quickwit-oss
U64 as displayed in the UI are wrong.
As spotted in #2011.
U64 field are very nice for users indexing ids. They can be a nicer solution than strings, for instance if we do grouping and the number of distinct values does not justify dicitonary encoding.
Javascript will deserialize their JSON value as a javascript Number, which has a limited precision. The number as displayed in the UI will hence be erroneous.
This limitation is not really a JSON issue (AFAIK JSON does not restrict the range of integer values), but is shared by many other programming language.
I am not entirely sure what we should do here tbh. :)
We also probably support string as an input for u64.
On serialize, a common solution to this problem is to add a string version of int fields.
"id": 23423432432424324,
"id_str": "23423432432424324"
But then we could have some collision problems. We could add a serialization option to serializes all ints as a string.
Twitter answered both types and settled on returning strings for integers. https://developer.twitter.com/en/docs/twitter-ids
My 2 cents:
We could add a serialization option to serializes all ints as a string.
If you need to expose an API and not just write an UI, I feel this is best.
Creating data duplication does not seem natural ("I send an id field but need to read an id_str". You can make the point that: "I send a integer value and get a string back" is not really better, at least it gets parsed)
It's something you could set globally in your config / add a specific endpoint for or override via an HTTP parameter.
We could use a custom parser in the browser https://github.com/josdejong/lossless-json Parsing is probably much slower though.
@PSeitz yeah but other users will suffer right? @saroh Elastic has u64 and i32 types nowdays right? Do they return string for u64?
@PSeitz yeah but other users will suffer right?
Yes, every browser UI would need to have some solution for that or may run into the same issue.
@saroh Elastic has u64 and i32 types nowdays right? Do they return string for u64?
i64 an i32 yes. They keep the data as is meaning the client may have surprises. The same issue is opened with Kibana https://github.com/elastic/kibana/issues/40183
