data icon indicating copy to clipboard operation
data copied to clipboard

Add documentation about custom Shuffle and Sharding DataPipe

Open ejguan opened this issue 2 years ago • 0 comments

📚 The doc issue

TorchData has a few special graph functions to handle Shuffle and Sharding DataPipe. But, we never document what is expected for those graph functions, which leads users to extend custom shuffle and sharding by diving into our code base.

We should add clear document about the expected methods attached to Shuffle or Sharding DataPipe.

This problem has been discussed in the #1081 as well, but I want to track documentation issue separately

Suggest a potential alternative/fix

No response

ejguan avatar Mar 21 '23 18:03 ejguan