floating_bar icon indicating copy to clipboard operation
floating_bar copied to clipboard

Overflow-handling strategies

Open 1011X opened this issue 1 year ago • 0 comments

floating_bar should decide on a default way to handle overflow for ops::Add and ops::Mul, and also include methods for different overflow-handling strategies so the user can decide which one is best for their use-case.

Strategies, in increasing order of complexity:

  • Lossless shift: shift out common powers of 2. Panic if not enough.
  • Lossy shift: shift out lowest bits until result fits in fraction field.
  • GCD reduce: divide by GCD. Panic if not enough.

The current plan is to use lossless shift as the default strategy since it is relatively quick and accurate, and floating_bar presents itself as a library focused on accuracy. The other behaviors will be available through a set of lossy_{add,mul}() and checked_{add,mul}() methods.

1011X avatar Feb 21 '23 22:02 1011X