ParallelUtilities.jl
ParallelUtilities.jl copied to clipboard
Fast and easy parallel mapreduce on HPC clusters
ParallelUtilities.jl
Parallel mapreduce and other helpful functions for HPC, meant primarily for embarassingly parallel operations that often require one to split up a list of tasks into subsections that may be processed on individual cores.
Installation
Install the package using
pkg> add ParallelUtilities
julia> using ParallelUtilities
Quick start
Just replace mapreduce
by pmapreduce
in your code and things should work the same.
julia> @everywhere f(x) = (sleep(1); x^2); # some expensive calculation
julia> nworkers()
2
julia> @time mapreduce(f, +, 1:10) # Serial
10.021436 seconds (40 allocations: 1.250 KiB)
385
julia> @time pmapreduce(f, +, 1:10) # Parallel
5.137051 seconds (863 allocations: 39.531 KiB)
385
Usage
See the documentation for examples and the API.