Julian Samaroo

Results 433 comments of Julian Samaroo

Instead of using `Threads.@threads` to parallelize this, you should use the `Threads.@spawn` approach but replace that with `Dagger.@spawn`. You almost always want to write the function you pass to Dagger...

Waiting on thunk completion and fetching thunk results will be implemented in #117

It seems like various failures when using Distributed will cause a cascade of failures in Dagger. We should try to wrap every call into Distributed APIs with a "rollback" function,...

Currently we make no guarantees about when and where work will be scheduled (unless constrained by the user via scopes, `single`, `procutil`, and similar mechanisms), only that input-output ordering is...

> Ok, so is the current strategy to just start a very large number of tasks on each processor then and let the task scheduler handle it? Basically, yeah. Until...

Can I ask why you want this, other than to ensure that new workers get utilized effectively? I'm thinking that I might not want to expose too many performance-focused knobs...

Worker-local work stealing can definitely be done in a straightforward manner by just using some locks and channels; the only complexity is satisfying the task's scope when stealing work (maybe...

There are actually two scheduling stages available in my mind: the first is in `schedule!`, which assigns tasks to their initial processors; however, a scheduled task is not actually executing...

Closes https://github.com/JuliaParallel/Dagger.jl/issues/165

I'm considering lower-bounding Julia to 1.7 going forward, so that we can easily work with atomics and use packages like ConcurrentCollections.jl and AtomicArrays.jl. If we do this, we'll do a...