returnn
returnn copied to clipboard
Documentation of Dataset API
in #294 there was a discussion about the Dataset API but it was rather concetrated on every single method of the class. Would be nice to have a high level overview of the Dataset class. With that I mean to specify the logical flow of how the dataset api is used.
It gets complicated to trace back where it all starts starts everytime.
Edit: Similiar to the execution guide.
Of course, the code itself is one source of documentation, i.e. how the dataset is actually used (in TFEngine/Runner). (RETURNN is not intended for users who do not want to read its code.)
And also this tool is also meant to be an example for the dataset API: dump-dataset
An even more high level documentation is our RETURNN tutorial, which also covers the dataset. (Did you check that?)
And otherwise, I think it then really boils down to individual functions.
But any of that can be improved of course. Maybe also some separate high level documentation. Feel free to make a PR about it.