progress icon indicating copy to clipboard operation
progress copied to clipboard

Reduce printing in tight loops

Open lavrentivs opened this issue 4 years ago • 7 comments

Printing is slow, which is an issue for big tight loops (~100k items, at 100 items/second). These changes reduce the printing to, at most, twice per second by default. Setting print_dt = -1 will preserve the previous behaviour.

lavrentivs avatar Jul 20 '20 01:07 lavrentivs

This should also help with https://github.com/verigak/progress/issues/56

lavrentivs avatar Jul 20 '20 01:07 lavrentivs

The latest commits caused a conflict to this PR. However it was incomplete as your change would only run on spinners, not bars. You will need to update writeln if you are still interested pursuing this.

verigak avatar Jul 20 '20 11:07 verigak

Updated. afaict all classes inherit from Infinite and there is no overwriting of writeln. In particular, this is the inheritance chain for a bar ChargingBar --> Bar --> Progress --> Infinite .

lavrentivs avatar Jul 20 '20 11:07 lavrentivs

Right, but when you submitted your PR, bars where using writeln while spinners etc were using write. In any case everything is unified now.

However there are still a number of issues with your PR:

  • print_dt should not be an init argument. It should either be a class attribute, or passed in kwargs. Please follow the conventions of the code.
  • What's the rationale for the initialization to this? self.start_ts - self.print_dt * 2
  • Also please follow the Python style conventions

verigak avatar Jul 20 '20 11:07 verigak

Regarding your comments:

  • print_dt moved as a class attribute but allow kwargs to change it.
  • self._prev_write = self.start_ts - self.print_dt * 2 will force the next call to self.writeln('') to actually print.
  • Can you point me out to which Python/code conventions I am breaking?

I've also added changes to force that the last line is printed (ending a bar with 99% would be strange).

Thanks for taking the time reviewing this PR :)

lavrentivs avatar Jul 20 '20 23:07 lavrentivs

I think it would be better to make print_dt a class attribute instead, like sma_window etc. Also I don't like the magic value of _prev_write. Why * 2 and not 1.5 or 3? I'd rather initialize it to 0 and then add a force=False argument in writeln to bypass the check when need it. Finally if you are going to keep track of _prev_line you might as well use it and not print at all if it hasn't canged (independent of print_dt)

verigak avatar Jul 29 '20 09:07 verigak

FWIW, not only printing is inefficient. There are floating point number miscalculations, combinded with a very short moving average window. Combined with NIH syndrome upstream, we keep this downstream patch for several years now.

praiskup avatar Jul 29 '20 11:07 praiskup