singa
singa copied to clipboard
Add a dataset module
Data loading is an important part of DL training, which could be slow and become a bottleneck if not implemented well. The tasks include
- implement dataset classes for common benchmark datasets to make them easy to access within SINGA (e.g., without manual downloading).
- implement common preprocessing operations
- implement parallel data loading for higher efficiency
Code from the data module may be reused. https://github.com/apache/singa/blob/master/python/singa/data.py
And https://github.com/apache/singa/blob/master/python/singa/image_tool.py