data
data copied to clipboard
Add documentation about custom Shuffle and Sharding DataPipe
📚 The doc issue
TorchData has a few special graph functions to handle Shuffle and Sharding DataPipe. But, we never document what is expected for those graph functions, which leads users to extend custom shuffle and sharding by diving into our code base.
We should add clear document about the expected methods attached to Shuffle or Sharding DataPipe.
This problem has been discussed in the #1081 as well, but I want to track documentation issue separately
Suggest a potential alternative/fix
No response