Generalized Channel Throttling

[teddyrendahl]

What's the problem this feature will solve?

High rate data can soak up a lot of CPU, especially if the PyDMChannel is subscribed to multidimensional data.

Describe the solution you'd like

I would like to see throttling done at the basePlugin level. Obviously there isn't that much you can do if you add a callback to a PV, but we can at least generally stop the UI from updating. There could be some way of indicating this on the channel URI itself: ca://My:Pv/max_rate=5

Right now you can do it on the PyDMImageView but I think the solution needs to be made more global. I would even suggest pydm has a global max redraw rate of about 30Hz (that can be modified by the user if they think they need it)

[klauer]

Would certain widgets need to opt-out of this behavior somehow? I don't think it'd be acceptable for a plot to drop data points, for example.

Perhaps there is something to be learned from how bluesky has been moving toward pagination of data, intake, etc..

[hhslepicka]

Sorry, my previous comment was wrong.. I thought this message was in another thread!

[hhslepicka]

I agree that there must be a way for widgets to opt-out of the throttling... We currently do that on the maxRedrawRate for plots and Image widgets. We should always allow users to go full rate if they have the need even if this means chewing an unreasonable amount of data.

[mattgibbs]

Its worth differentiating between a maximum redraw rate and a maximum data rate. As @klauer notes, you don't want to drop data in plots, so the way we do the throttling there is only redraw on a timer, but keep updating the data buffer at the full data rate. We probably want both redraw and data throttling.

I think CPU usage is usually driven by redrawing, not by consuming data from channels, so implementing redraw rate limits on widgets is probably higher priority. A global maximum redraw rate sounds like a nice idea too, although I wonder how tricky the implementation will be...

[teddyrendahl]

A global maximum redraw rate sounds like a nice idea too, although I wonder how tricky the implementation will be...

I believe this would be fairly simple if you run your event loop. It might be then hard to have a widget update faster than the "global" rate however. In the past we have been a little wary to dive down this rabbit hole, but now that we are dropping Python2, we could use a standard library option with asyncio