enhancements icon indicating copy to clipboard operation
enhancements copied to clipboard

[Core] Add a new interface for submitting actor tasks in batches (Batch Remote)

Open larrylian opened this issue 1 year ago • 9 comments

Core Motivation:

  1. Improve the performance of batch calling ActorTask.
  2. Implement Ray's native collective communication library through this interface. -- This will be discussed in a new REP afterwards.

Current situation of batch calling actor tasks:

actors = [WorkerActor.remote() for _ in range(400)]

# This loop's repeated invocation actually wastes a lot of performance.
for actor in actors:
  actor.compute.remote(args)

Using the new Batch Remote API:

actors = [WorkerActor.remote() for _ in range(400)]

# Calling it only once can greatly improve performance.  
Plan 1
batch_remote_handle = ray.experimental.batch_remote(actors)
batch_remote_handle.compute.remote(args)

Plan 2
batch_remote_handle = ray.experimental.BatchRemoteHandle(actors)
batch_remote_handle.compute.remote(args)

The Batch Remote API can save the following performance costs(The N is the number of Actors):

  1. Reduce (N-1) times of parameter serialization performance time.
  2. Reduce (N-1) times of putting parameter into object store performance time for scenarios with large parameters.
  3. Reduce (N-1) times of python and C++ execution layer switching and repeated parameter verification performance time.

[WIP][Core]Add batch remote api for batch submit actor task https://github.com/ray-project/ray/pull/35597

larrylian avatar May 13 '23 14:05 larrylian