vision icon indicating copy to clipboard operation
vision copied to clipboard

Classification references does not work without distributed setup

Open pmeier opened this issue 3 years ago • 2 comments

If you don't set the respective env vars

https://github.com/pytorch/vision/blob/d5bd8b728f14c33b339fc45c90ca39be339bce3f/references/classification/utils.py#L255-L258

training will not be distributed and in turn the backend will not be initialized. However, during evaluation we check

https://github.com/pytorch/vision/blob/d5bd8b728f14c33b339fc45c90ca39be339bce3f/references/classification/train.py#L88

unguarded, which then fails with

RuntimeError: Default process group has not been initialized, please make sure to call init_process_group.

cc @datumbox

pmeier avatar Sep 01 '22 08:09 pmeier

Same for segmentation:

https://github.com/pytorch/vision/blob/cac4e228c9ca9e7564cb34406e7ebccfdd736976/references/segmentation/train.py#L84

pmeier avatar Sep 12 '22 15:09 pmeier

I think this case is implicitly guarded on https://github.com/pytorch/vision/blob/d5bd8b728f14c33b339fc45c90ca39be339bce3f/references/classification/train.py#L87

since len(data_loader.dataset) != num_processed_samples shouldn't be true on non-distributed setting.

Do you get the error during non-distributed training @pmeier ?

YosuaMichael avatar Sep 14 '22 16:09 YosuaMichael