Kevin Tse
Kevin Tse
The argument `input_col` should allow you to do that with [`map`](https://pytorch.org/data/main/generated/torchdata.datapipes.iter.Mapper.html?highlight=mapper#torchdata.datapipes.iter.Mapper). You need to download the latest version though. ```python dp = IterableWrapper([("a", 1), ("b", 2)]).map(fn=lambda char: char + char,...
Adding `map_tuple` and `map_dict` should be easy. It will mostly just be wrappers around the existing `map` implementation (i.e. passing `key` of dict to `input_col` and `idx` of tuple to...
@msaroufim I think the API/functionality discussed here is different from `itertools`, but nonetheless I see your point. I just opened #756 to discuss.
Potential duplicate issues of: * #649 * #655 * #661 We should keep one and close the other ones?
> @NicolasHug is there a setup script I need to run beforehand? https://github.com/pytorch/data/runs/7868534954?check_suite_focus=true#step:9:23 That only works on the internal AI cluster. I will message you more details.
> I personally think so as fast forward would be the backup way to run snapshot. It would be better to use __getstate__ to consolidate the state of DataPipe and...
> If you call `dl.shutdown()` at the end, is the problem still persistent? I can confirm that it does fix it. I think the issue is that `reading_service.finalize()` never get...
CI error unrelated
@nivekt has imported this pull request. If you are a Facebook employee, you can view this diff [on Phabricator](https://www.internalfb.com/diff/D38868731).
> One more though, we deepcopy RS, but we pickle-unpickle DP. I think we need to pick consistent approach. Can you elaborate on this? What make a consistent approach matter...