whylogs icon indicating copy to clipboard operation
whylogs copied to clipboard

Need to handle and document overflow/underflow in metric components input and serialization

Open jamie256 opened this issue 2 years ago • 0 comments

Description

When metrics are serialized, it is possible that the values in python are out of range with their message format's range: e.g. a python int value can be serialized in a metric component "n" field which maps to protobuf int64 value.

There are a few ways we could encounter overflow/underflow:

  1. Individual input values are integer type and some values might be out of range.
  2. operations which sum or merge values can result in overflow: e.g counts.

The places this becomes a problem is where the range is supported by python but not by the underlying libraries that implement statistics, when profiling, merging or during serialization (protobuf fields).

jamie256 avatar Sep 01 '22 18:09 jamie256