ThreadPools.jl
ThreadPools.jl copied to clipboard
Feasibility of using ThreadPools for I/O heavy operations.
I'm trying to use this library to parallelise a few I/O calls, Does it makes sense to use this library to work with I/O tasks in the following way ?
System Threads:
julia --threads 5
API Function:
function simulate_api_call()
println("API call started with Thread : $(Threads.threadid())")
# Simulate a long-running API call
# response_task = @async HTTP.get("https://randomuser.me/api/?results=50000")
response_task = Threads.@spawn HTTP.get("https://randomuser.me/api/?results=50000")
println("API call completed with Thread : $(Threads.threadid())")
return response_task
end
Usage of ThreadPools:
function test_spawn_3()
count = Threads.nthreads() - 1
results = Vector{Task}(under, count)
ThreadPools.qbforeach(1:count) do i
results[i] = simulate_api_call()
end
responses = fetch.(results)
return responses
end