ThreadPools.jl icon indicating copy to clipboard operation
ThreadPools.jl copied to clipboard

Feasibility of using ThreadPools for I/O heavy operations.

Open chamikabm opened this issue 1 year ago • 0 comments

I'm trying to use this library to parallelise a few I/O calls, Does it makes sense to use this library to work with I/O tasks in the following way ?

System Threads:

julia --threads 5

API Function:

function simulate_api_call()
     println("API call started with Thread : $(Threads.threadid())")

     # Simulate a long-running API call
     # response_task = @async HTTP.get("https://randomuser.me/api/?results=50000")

     response_task = Threads.@spawn HTTP.get("https://randomuser.me/api/?results=50000")

     println("API call completed with Thread : $(Threads.threadid())")

     return response_task
 end

Usage of ThreadPools:

function test_spawn_3()
    count =  Threads.nthreads() - 1
    results = Vector{Task}(under, count)

    ThreadPools.qbforeach(1:count) do i
        results[i] = simulate_api_call()
    end

    responses = fetch.(results)

    return responses
end

chamikabm avatar Nov 28 '23 21:11 chamikabm