Meta-DETR icon indicating copy to clipboard operation
Meta-DETR copied to clipboard

How much time it takes to train an epoch when using one v100?

Open IceHowe opened this issue 2 years ago • 8 comments

How much time it takes to train an epoch when using one v100? Why did I train an epoch with one 2080TI for a day(around 24 hours) ? Is there anything wrong with it? Or is that the reality?

IceHowe avatar Jun 22 '22 08:06 IceHowe

And I've changed the ResNet from 101 to 50, and multiscale training has only used 480 and 512

IceHowe avatar Jun 22 '22 09:06 IceHowe

Thanks for your interest. Please check if you correctly follow our training scripts.

On Pascal VOC Split 1, one training epoch takes around 19-20 minutes based on my training logs.

ZhangGongjie avatar Jun 22 '22 09:06 ZhangGongjie

Thanks for your interest. Please check if you correctly follow our training scripts.

On Pascal VOC Split 1, one training epoch takes around 19-20 minutes based on my training logs.

Thanks for your reply. In your paper, I see you use 8 × v100, how many batchsize do you set? And I want to konw the time one epoch when training on MSCOCO. I set batchsize=4, and use one 2080TI on MSCOCO to training, which takes one day.

时间

IceHowe avatar Jun 22 '22 11:06 IceHowe

I also meet this question, and I use one Tesla V100 GPU(32G), and I change batch_size from 4 to 8, it seems like need 11 hours to train one epoch. 捕获

YAOSL98 avatar Oct 29 '22 10:10 YAOSL98

Thanks for your interest. Please check if you correctly follow our training scripts. On Pascal VOC Split 1, one training epoch takes around 19-20 minutes based on my training logs.

Thanks for your reply. In your paper, I see you use 8 × v100, how many batchsize do you set? And I want to konw the time one epoch when training on MSCOCO. I set batchsize=4, and use one 2080TI on MSCOCO to training, which takes one day.

时间

Is it all very slow during training? The 3060Ti I use is too slow. Can I add your contact information so we can discuss related issues.

Caltech-Z avatar Dec 07 '22 07:12 Caltech-Z

您好!您发给我的信件已经收到,我将尽快处理 

YAOSL98 avatar Dec 07 '22 07:12 YAOSL98