ray_shuffling_data_loader icon indicating copy to clipboard operation
ray_shuffling_data_loader copied to clipboard

Published 20 hours ago • verified publisher icon ray-project

[Testing] Add unit tests.

Open clarkzinzow opened this issue 4 years ago • 1 comments

Add unit tests for the following:

  1. Shuffler (shuffle.py)
  2. BatchQueue (batch_queue.py)
  3. ShufflingDataset (dataset.py)
  4. TorchShufflingDataset (torch_dataset.py)

Implementation Notes

  • ShufflingDataset and TorchShufflingDataset testing could initially be as simple as running the example in the main block of dataset.py and torch_dataset.py, respectively, although these don't do much to assert correctness of the yielded batches.
  • See these tests for easily portable examples for testing BatchQueue.

clarkzinzow avatar Jul 13 '21 07:07 clarkzinzow

I'll start with setting up CI and running examples, and then will add unit tests that also assert correctness.

  • [x] Shuffler (#22)
  • [x] BatchQueue (#21)
  • [x] ShufflingDataset (#19, #20)
  • [x] TorchShufflingDataset (#19, #20)

krfricke avatar Jul 13 '21 08:07 krfricke