xenova
Ability to Pause model / throttle CPU
Feature request
The problem I'm facing, is that a running model within an extension, even on the background script (service worker) - it's blocking other actions within the extension. This means we cannot interact with any extension pages whilst a model is running (for example extension options/popup/new tab).
This is weird, as from what I understand the whole point of worker threads is to not block UI threads.. https://developer.chrome.com/docs/extensions/develop/migrate/to-service-workers
A fix could be the ability to pause the model (perhaps you can provide the slice time when you run), then we could sleep / similar after each time slice, allowing the UI / other threads can catch up.
Perhaps some throttling of the CPU may also work.
https://github.com/xenova/transformers.js/issues/341 is a little similar (except cancellation)
Motivation
Long-running models are not very use-able right now within extensions, is there's also a UI - this feature would aim to improve this.
Your contribution
I could perhaps look into this, but some discussion first Perhaps once I have a better understanding of any current technical limitations.
