data icon indicating copy to clipboard operation
data copied to clipboard

Add option to disable random seed sharding and syntonization for DistributedReadingService

Open ejguan opened this issue 2 years ago • 0 comments

🚀 The feature

This is a customer request for inference, no synchronization is needed because no all_reduce is needed. And, normally inference doesn't require shuffling and requires high throughput. Sharing random seed becomes redundant in this scenario.

We might want to add two more arguments to enable/disable synchronization.

Motivation, pitch

Provide better performance for inference using DataPipe + DataLoader2

Alternatives

Add a special ReadingService for distributed inference.

Additional context

No response

ejguan avatar Feb 24 '23 21:02 ejguan