enhancements
enhancements copied to clipboard
[Core] Add a new interface for submitting actor tasks in batches (Batch Remote)
Core Motivation:
- Improve the performance of batch calling ActorTask.
- Implement Ray's native collective communication library through this interface. -- This will be discussed in a new REP afterwards.
Current situation of batch calling actor tasks:
actors = [WorkerActor.remote() for _ in range(400)]
# This loop's repeated invocation actually wastes a lot of performance.
for actor in actors:
actor.compute.remote(args)
Using the new Batch Remote API:
actors = [WorkerActor.remote() for _ in range(400)]
# Calling it only once can greatly improve performance.
Plan 1
batch_remote_handle = ray.experimental.batch_remote(actors)
batch_remote_handle.compute.remote(args)
Plan 2
batch_remote_handle = ray.experimental.BatchRemoteHandle(actors)
batch_remote_handle.compute.remote(args)
The Batch Remote API can save the following performance costs(The N is the number of Actors):
- Reduce (N-1) times of parameter serialization performance time.
- Reduce (N-1) times of putting parameter into object store performance time for scenarios with large parameters.
- Reduce (N-1) times of python and C++ execution layer switching and repeated parameter verification performance time.
[WIP][Core]Add batch remote api for batch submit actor task https://github.com/ray-project/ray/pull/35597