ert icon indicating copy to clipboard operation
ert copied to clipboard

Make callback.py async

Open berland opened this issue 1 year ago • 2 comments

When the legacy jobqueue and the associated C-drivers are removed, make forward_model_ok callbacks.py an async function in order to mitigate the downsides of running data internalization as sync code. Yield to async as often as possible in this file.

This was done in https://github.com/equinor/ert/commit/a767ad9c19f5488032825e4df38cdcdd5a39db91 with measurable positive effect.

berland avatar Feb 26 '24 13:02 berland

Data internalisation is a CPU-heavy task, so asyncio won't gain much. (Asyncio is really only useful in operations when you are waiting a lot, like in networking or what we are using it for in Scheduler.)

We should resurrect that attempt at offloading forward_model_ok to the cluster.

pinkwah avatar Feb 26 '24 20:02 pinkwah

This function is already called in an async context, so the "mitigation" is only about yielding a little more often.

berland avatar Feb 27 '24 06:02 berland