dist_tuto.pth

dist_tuto.pth copied to clipboard

Whether we maintain two same models in two process

1

https://github.com/seba-1511/dist_tuto.pth/blob/82ee2a360cb8670dd2724913e0200939d370aa56/train_dist.py#L106 Whether it isn't useful in any real practice? I think eventually we only need one model How I can know which GPU place each data replicas and each model?

LukeLIN-web

train_dist.py incompatible with new PyTorch

5

When I tried to run train_dist.py, I ran into multiple errors that seem to be due to the fact that this code is written on older versions of Python and...

LUKELIEM

My Code： ``` """run.py:""" #!/usr/bin/env python import os import sys import torch import torch.distributed as dist import time from torch.multiprocessing import Process # """Blocking point-to-point communication.""" # def run(rank, size):...

Weigaa

Sorry but I can not find out the similar documents with c++ language. Please give me some example with distributed libtorch c++. Thanks,

ph0123

so i run train_dist.py and add print(param) under if section ` if type(param) is torch.Tensor: print(param) dist.all_reduce(param.grad.data, op=dist.reduce_op.SUM, group=0) ` param never printed which means all_reduce never called.

mambo06