Dagger.jl icon indicating copy to clipboard operation
Dagger.jl copied to clipboard

Dagger.nworkers API?

Open tkf opened this issue 4 years ago • 2 comments

Does Dagger have some kind of "nworkers" function? I'm thinking something like sum(fetch, map(id -> remotecall(Threads.nthreads, id), Distributed.workers())) but more efficient/cached. Or maybe it should be called something like ncpuworkers to make it less confusing when you use GPU?

tkf avatar Mar 04 '21 01:03 tkf

I've been using sum([length(Dagger.get_processors(OSProc(id))) for id in workers()]) to calculate total processors. You should filter the resuts of get_processors to only Dagger.ThreadProc instances to get the number of threads. OSProc(id) should be cached on each node, so it should be reasonably fast.

jpsamaroo avatar Mar 04 '21 12:03 jpsamaroo

Great! This seems to do what I wanted: sum(1 for id in workers() for c in Dagger.get_processors(OSProc(id)) if c isa Dagger.ThreadProc).

I'm OK with writing this code on my end, but I guess it's a bit tricky to come up with? I wonder if it makes sense to provide a function like this?

"""
    countprocs([ProcessorType]) -> number::Int

Count the number of available Dagger processors.
"""
countprocs(T::Type{<:Processor} = Processor) =
    sum(1 for id in workers() for c in get_processors(OSProc(id)) if c isa T)

Maybe the best strategy here is to wait for more people requesting something similar, so that the API would be more constrained? Anyway, please feel free to close or keep this issue.

tkf avatar Mar 04 '21 22:03 tkf