deeplake icon indicating copy to clipboard operation
deeplake copied to clipboard

Published 20 hours ago • verified publisher icon activeloopai

[FEATURE] API for sampler

Open Phoveran opened this issue 3 years ago • 3 comments

Thanks for your remarkable contribution! Is there any API for sampler in pytorch when getting the dataloader? e.g. torch.utils.data.WeightedRandomSampler

Phoveran avatar May 31 '22 02:05 Phoveran

@Phoveran thanks for raising the issue, we are reimplementing the pytorch dataloader and adding the sampler strategies as well. Please stay tuned!

davidbuniat avatar May 31 '22 02:05 davidbuniat

Hi, is there a way to request a subset of a dataset? For example, the first 256 samples?

adolfogc avatar May 31 '22 08:05 adolfogc

yes you should be able to ds[0:256].pytorch(...)

davidbuniat avatar May 31 '22 15:05 davidbuniat

Hi @Phoveran We added support for sampling using our Tensor Query Language (TQL) described here. Unfortunately, we decided not to support native Pytorch Samplers, but functionally, you should be able to satisfy your use-case using TQL.

istranic avatar Jan 04 '23 17:01 istranic