image-captioning-DLCT icon indicating copy to clipboard operation
image-captioning-DLCT copied to clipboard

关于训练阶段Shuffle参数设置问题

Open yanjialele opened this issue 2 years ago • 2 comments

作者您好,在服务器上复现您代码的时候发现如下报错 Traceback (most recent call last): File "train.py", line 328, in dict_dataloader_train = DataLoader(dict_dataset_train, batch_size=args.rl_batch_size // 5, shuffle=True, File "/mnt/hdd1/alluser/yanjialuo/image-captioning-DLCT/data/init.py", line 8, in init super(DataLoader, self).init(dataset, *args, collate_fn=dataset.collate_fn(), **kwargs) File "/mnt/ssd1/anaconda3/envs/yjl/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 228, in init raise ValueError( ValueError: DataLoader with IterableDataset: expected unspecified shuffle option, but got shuffle=True 我检查了一下源码,发现在train.py的328行 dict_dataloader_train = DataLoader(dict_dataset_train, batch_size=args.rl_batch_size // 5, shuffle=True, num_workers=args.workers) 若将shuffle置为False,能够运行,请问作者您遇到过上述的问题么?期待您的回复

yanjialele avatar Nov 21 '21 02:11 yanjialele

您好,这个问题主要是pytorch版本导致的,pytorch1.10左右的版本开始会把代码里的dataset识别为IterableDataset的子类,所以建议用之前的版本,最好是用1.2,当然1.7之前应该也都可以

luo3300612 avatar Dec 06 '21 06:12 luo3300612

@luo3300612 Hello, excuse me, I want to ask you a question, My cache keeps increasing when I run it and eventually the model gets stuck. I am expecting your reply.

Baixiaobai201619707 avatar Apr 05 '22 15:04 Baixiaobai201619707