stylegan-t icon indicating copy to clipboard operation
stylegan-t copied to clipboard

run in ubuntu

Open lcxsnow opened this issue 1 year ago • 2 comments

when I run the code in ubuntu, which has more than one gpu, it appeared the below error:

(pytorch) m11113013@htsc-Server3:~/ProjectCode/MasterProject2$ python -m torch.distributed.run --standalone --nproc_per_node 1 train.py --outdir ./training-runs/ --cfg lite --data /dataset/MSCOCO/mscoco.zip --img-resolution 64 --batch 8 --batch-gpu 2 --kimg 25000 --metrics fid50k_full Error: --metrics can only contain the following values: ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 4694) of binary: /home/m11113013/miniconda3/envs/pytorch/bin/python Traceback (most recent call last): File "/home/m11113013/miniconda3/envs/pytorch/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/home/m11113013/miniconda3/envs/pytorch/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/m11113013/miniconda3/envs/pytorch/lib/python3.8/site-packages/torch/distributed/run.py", line 765, in main() File "/home/m11113013/miniconda3/envs/pytorch/lib/python3.8/site-packages/torch/distributed/elastic/multiprocessing/errors/init.py", line 345, in wrapper return f(*args, **kwargs) File "/home/m11113013/miniconda3/envs/pytorch/lib/python3.8/site-packages/torch/distributed/run.py", line 761, in main run(args) File "/home/m11113013/miniconda3/envs/pytorch/lib/python3.8/site-packages/torch/distributed/run.py", line 752, in run elastic_launch( File "/home/m11113013/miniconda3/envs/pytorch/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 131, in call return launch_agent(self._config, self._entrypoint, list(args)) File "/home/m11113013/miniconda3/envs/pytorch/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 245, in launch_agent raise ChildFailedError( torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

train.py FAILED

Failures: <NO_OTHER_FAILURES>

Root Cause (first observed failure): [0]: time : 2023-04-30_22:44:16 host : htsc-Server3 rank : 0 (local_rank: 0) exitcode : 1 (pid: 4694) error_file: <N/A> traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html

When running on Window, it appeared the below errors:

(pytorch) C:\Users\snow\Desktop\python project\stylegan-t-main>python -m torch.distributed.run --standalone --nproc_per_node 1 train.py --outdir ./training-runs/ --cfg lite --data ./data/coco_val256.zip --img-resolution 256 --batch 128 --batch-gpu 2 --kimg 25000 --metrics fid50k_full NOTE: Redirects are currently not supported in Windows or MacOs. Traceback (most recent call last): File "train.py", line 25, in from training import training_loop File "C:\Users\snow\Desktop\python project\stylegan-t-main\training\training_loop.py", line 32, in from metrics import metric_main File "C:\Users\snow\Desktop\python project\stylegan-t-main\metrics\metric_main.py", line 20, in from metrics import precision_recall File "C:\Users\snow\Desktop\python project\stylegan-t-main\metrics\precision_recall.py", line 36, in def compute_pr(opts, max_real: int, num_gen: int, nhood_size: int, row_batch_size: int, col_batch_size: int) -> tuple[float, float]: TypeError: 'type' object is not subscriptable ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 8652) of binary: C:\Users\snow\miniconda3\envs\pytorch\python.exe Traceback (most recent call last): File "C:\Users\snow\miniconda3\envs\pytorch\lib\runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "C:\Users\snow\miniconda3\envs\pytorch\lib\runpy.py", line 85, in run_code exec(code, run_globals) File "C:\Users\snow\miniconda3\envs\pytorch\lib\site-packages\torch\distributed\run.py", line 728, in main() File "C:\Users\snow\miniconda3\envs\pytorch\lib\site-packages\torch\distributed\elastic\multiprocessing\errors_init.py", line 345, in wrapper return f(*args, **kwargs) File "C:\Users\snow\miniconda3\envs\pytorch\lib\site-packages\torch\distributed\run.py", line 724, in main run(args) File "C:\Users\snow\miniconda3\envs\pytorch\lib\site-packages\torch\distributed\run.py", line 718, in run )(*cmd_args) File "C:\Users\snow\miniconda3\envs\pytorch\lib\site-packages\torch\distributed\launcher\api.py", line 131, in call return launch_agent(self._config, self._entrypoint, list(args)) File "C:\Users\snow\miniconda3\envs\pytorch\lib\site-packages\torch\distributed\launcher\api.py", line 247, in launch_agent failures=result.failures, torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

train.py FAILED

Failures: <NO_OTHER_FAILURES>

Root Cause (first observed failure): [0]: time : 2023-05-01_13:33:36 host : DESKTOP-FIA0K2K rank : 0 (local_rank: 0) exitcode : 1 (pid: 8652) error_file: <N/A> traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html

lcxsnow avatar May 01 '23 05:05 lcxsnow

I am also getting the same error. Did you get any reply yet?

snd-ml avatar May 15 '23 11:05 snd-ml

+1

I am also getting the same error. Did you get any reply yet?

ADAMS12345678 avatar Oct 20 '23 09:10 ADAMS12345678