faust icon indicating copy to clipboard operation
faust copied to clipboard

Repeated Timer Overlap Warning with SetTable

Open tapple opened this issue 4 years ago • 5 comments

Checklist

  • [x] I have included information about relevant versions
  • [x] I have verified that the issue persists when using the master branch of Faust.

Steps to reproduce

Run this demo worker like python bugdemo.py worker:

import faust
app = faust.App('example', broker='kafka://')
set_table_1 = app.SetTable(name='set_table_1')
set_table_2 = app.SetTable(name='set_table_2')
# set_table_3 = app.SetTable(name='set_table_3')
# set_table_4 = app.SetTable(name='set_table_4')
# set_table_5 = app.SetTable(name='set_table_5')

if __name__ == '__main__':
    app.main()

Expected behavior

The worker should do absolutely nothing. It has no tasks or agents

Actual behavior

I get this warning spam repeatedly in the logs:

[2020-03-12 11:14:29,644] [86917] [WARNING] Timer SetManager.flush is overlapping (interval=2.0 runtime=2.0014956910163164) 
[2020-03-12 11:14:33,644] [86917] [WARNING] Timer SetManager.flush is overlapping (interval=2.0 runtime=2.000451576896012) 
[2020-03-12 11:14:33,644] [86917] [WARNING] Timer SetManager.flush is overlapping (interval=2.0 runtime=2.0007298621349037) 

It continues until I kill the worker. I get the warning more frequently with more SetTables, and less frequently with only 1 SetTable, and never with 0 SetTables

Versions

  • Python version 3.6.4
  • Faust version 1.10.4
  • Operating system mac 10.15.3
  • Kafka version 2.4.0

tapple avatar Mar 12 '20 16:03 tapple

SetManager.flush seems to be here: https://github.com/robinhood/faust/blob/master/faust/tables/objects.py#L145-L149

tapple avatar Mar 12 '20 16:03 tapple

SetManager.flush seems to be here: https://github.com/robinhood/faust/blob/master/faust/tables/objects.py#L145-L149

Is that self.sleep(sleeptime) supposed to be there? I feel like there shouldn't be a sleep inside a timer

tapple avatar Mar 12 '20 16:03 tapple

If I comment out await self.sleep(sleep_time), the warning goes away, and cpu usage is still mostly idle, so, I think this might be the correct fix

tapple avatar Mar 12 '20 16:03 tapple

Any news regarding this?

2phost avatar Sep 03 '21 16:09 2phost

This project appears to have been abandoned.

You might want to check out the fork of this project - https://github.com/faust-streaming/faust

bobh66 avatar Sep 04 '21 15:09 bobh66