rich icon indicating copy to clipboard operation
rich copied to clipboard

Progress: Added an option to TimeRemainingColumn to estimate the speed using an exponential moving average

Open Akulen opened this issue 8 months ago • 0 comments

Type of changes

  • [ ] Bug fix
  • [x] New feature
  • [x] Documentation / docstrings
  • [x] Tests
  • [ ] Other

Checklist

  • [x] I've run the latest black with default args on new code.
  • [x] I've updated CHANGELOG.md and CONTRIBUTORS.md where appropriate.
  • [x] I've added tests for new code.
  • [x] I accept that @willmcgugan may be pedantic in the code review.

Description

I added a new attribute speed_ema to the Task class that stores a speed estimate using an Exponential Moving Average (EMA). It is updated whenever progress.update or progress.advance are called with new completed steps.

In addition, a parameter speed_estimate_alpha was added to Progress to control the EMA. The base formula used was: $$S_{i+1}=\left(\frac{\alpha}{s} + \frac{1-\alpha}{S_{i}}\right)^{-1}$$ where we do an harmonic weighted average of the speed, or a weighted average of the estimated remaining time. By interpolating this formula extension to $k$ consecutive speed updates with the same speed, we get the general formula $$S_{i+x} = \left(\frac{1 - (1 - \alpha)^x}{s} + \frac{(1-\alpha)^x}{S_{i}}\right)^{-1}$$ for an arbitrary $x>0$

At the start, as we have no previous estimate, so $S_0 = s_0$, and for the first few updates, we use a larger $\alpha$ so that the first few speed estimates do not get a disproportionate weight. In practice, we use $\alpha=max(\frac{1}{i+1}, \text{speed\_estimate\_alpha})$

Extra

The variable names are not ideal, being: speed_estimate_alpha, speed_ema, and time_remaining_ema, with ema abbreviating exponential_moving_average

I also updated pycvn in the pre-commit hooks to work in python 3.12

Akulen avatar Jun 25 '24 09:06 Akulen