Sjoerd de Vries

Results 93 comments of Sjoerd de Vries

There is now an InProcessAssistant class.

Instead of communicating to the Dask scheduler, a worker could also try reach the Dask client inside the original assistant. In that case, use the same Dask mechanism as https://github.com/sjdv1982/seamless/issues/219,...

Non-local nested transformation now works for a "local" mini assistant (devel), by setting the Seamless assistant IP address to "localhost" inside the mini-assistant-devel Docker image. (the meaning of "local" is...

It seems that IPFS has a 1-2 MiB chunk size max

Also add an API where the assistant is polled for stream/log updates continuously, and display them correctly with tqdm. Similar to Dask `progress`.

Earlier feature descriptions: ```text - Support transformation meta (see old jobless), progress/prelim, logs, hardcancel, etc. First add this as a stdout stream to run-transformation, then store these things in mini-assistant...

Also ponder similarities with `ContextPool` and `TransformationPool`

Note that it should be tested with seamless-devel. Keeping `gfortran` inside the main Docker image is not really justified. Must be marked in the `test-list.txt`, also for the low level