Jack BAI

https://www.jackgethome.com/

Berkeley, CA VSPA @ UC Berkeley; MSCS @ UIUC

Results 6 comments of


Jack BAI

你们用的显存都是多大，为啥我12G显存 batch_size 要设置成2才不会oom

神他喵加钱,改参数好哇

RuntimeError: module must have its parameters and buffers on device cuda:0 (device_ids[0]) but found one of them on device: cuda:1

https://pytorch.org/docs/stable/generated/torch.nn.DataParallel.html

RuntimeError: module must have its parameters and buffers on device cuda:0 (device_ids[0]) but found one of them on device: cuda:1

Further Hint: 只要把`device='cuda'`改成`device=cuda:0`就可以解决了

训练效率问题

我单机四块V100,用的DistributedDataParallel+Apex,10秒一个epoch,5m的语料库(0.1B)一小时基本拟合完成.

master分支，能否提供一下train.json的样例

已经加上去了

能问一下你用的环境的tensorflow版本吗？

这个项目主要是PyTorch做的叭,可以说下哪里用到了tf吗?