Vitaly Fedyunin
Vitaly Fedyunin
In addition to it, DLv2 should guarantee that `finalize` of `ReadingService` is called, even if Exception raised during iterations.
Please switch the order of inputs 'pattern' -> 'new name' looks more natural
In my opinion it makes sense to have two datapipes: `pattern_filter_keys` -> takes patterns, throws away all missmatch keys #406 and `pattern_rename_keys` -> takes pattern->new_name dictionary and renames keys accordingly....
@VitalyFedyunin has imported this pull request. If you are a Meta employee, you can view this diff [on Phabricator](https://www.internalfb.com/diff/D40280665).
That's one of the options. Also requires join timeouts and errors propagation.
@VitalyFedyunin has imported this pull request. If you are a Meta employee, you can view this diff [on Phabricator](https://www.internalfb.com/diff/D40308358).
@VitalyFedyunin has imported this pull request. If you are a Meta employee, you can view this diff [on Phabricator](https://www.internalfb.com/diff/D40308358).
We should put this information into the ReadingService interface file too. To make sure that RS developers know about potential deep copy.
One more though, we deepcopy RS, but we pickle-unpickle DP. I think we need to pick consistent approach.
Rebased, would be nice to land to clean torchdata's dependency tests.