whylogs
whylogs copied to clipboard
Need to handle and document overflow/underflow in metric components input and serialization
Description
When metrics are serialized, it is possible that the values in python are out of range with their message format's range: e.g. a python int
value can be serialized in a metric component "n" field which maps to protobuf int64 value.
There are a few ways we could encounter overflow/underflow:
- Individual input values are integer type and some values might be out of range.
- operations which sum or merge values can result in overflow: e.g counts.
The places this becomes a problem is where the range is supported by python but not by the underlying libraries that implement statistics, when profiling, merging or during serialization (protobuf fields).