How to train

[GODjlc]

here is the error /home/nuist/anaconda3/envs/siammask/lib/python3.6/site-packages/torch/optim/lr_scheduler.py:100: UserWarning: Detected call of lr_scheduler.step()beforeoptimizer.step(). In PyTorch 1.1.0 and later, you should call them in the opposite order: optimizer.step()beforelr_scheduler.step()`. Failure to do this will result in PyTorch skipping the first value of the learning rate schedule.See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate "https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate", UserWarning) Traceback (most recent call last): File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/train_siammask.py", line 292, in main() File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/train_siammask.py", line 160, in main assert os.path.isfile(args.resume), '{} is not a valid file'.format(args.resume) AssertionError: snapshot/checkpoint_e9.pth is not a valid file test snapshot 1 /home/nuist 20 on dataset VOT2018 with 4 gpus. [2020-02-02 19:55:54,102-rk0-test.py#555] Namespace(arch='Custom', config='config.json', cpu=False, dataset='VOT2018', debug=False, gt=False, log='log_test.txt', mask=True, refine=False, resume='snapshot/checkpoint_e15.pth', save_mask=False, video='', visualization=False) [2020-02-02 19:55:54,113-rk0-test.py#555] Namespace(arch='Custom', config='config.json', cpu=False, dataset='VOT2018', debug=False, gt=False, log='log_test.txt', mask=True, refine=False, resume='snapshot/checkpoint_e7.pth', save_mask=False, video='', visualization=False) [2020-02-02 19:55:54,114-rk0-test.py#555] Namespace(arch='Custom', config='config.json', cpu=False, dataset='VOT2018', debug=False, gt=False, log='log_test.txt', mask=True, refine=False, resume='snapshot/checkpoint_e20.pth', save_mask=False, video='', visualization=False) [2020-02-02 19:55:54,248-rk0-test.py#555] Namespace(arch='Custom', config='config.json', cpu=False, dataset='VOT2018', debug=False, gt=False, log='log_test.txt', mask=True, refine=False, resume='snapshot/checkpoint_e6.pth', save_mask=False, video='', visualization=False) [2020-02-02 19:55:54,266-rk0-test.py#555] Namespace(arch='Custom', config='config.json', cpu=False, dataset='VOT2018', debug=False, gt=False, log='log_test.txt', mask=True, refine=False, resume='snapshot/checkpoint_e8.pth', save_mask=False, video='', visualization=False) [2020-02-02 19:55:54,275-rk0-test.py#555] Namespace(arch='Custom', config='config.json', cpu=False, dataset='VOT2018', debug=False, gt=False, log='log_test.txt', mask=True, refine=False, resume='snapshot/checkpoint_e10.pth', save_mask=False, video='', visualization=False) [2020-02-02 19:55:54,277-rk0-test.py#555] Namespace(arch='Custom', config='config.json', cpu=False, dataset='VOT2018', debug=False, gt=False, log='log_test.txt', mask=True, refine=False, resume='snapshot/checkpoint_e2.pth', save_mask=False, video='', visualization=False) [2020-02-02 19:55:54,287-rk0-test.py#555] Namespace(arch='Custom', config='config.json', cpu=False, dataset='VOT2018', debug=False, gt=False, log='log_test.txt', mask=True, refine=False, resume='snapshot/checkpoint_e12.pth', save_mask=False, video='', visualization=False) [2020-02-02 19:55:54,331-rk0-test.py#555] Namespace(arch='Custom', config='config.json', cpu=False, dataset='VOT2018', debug=False, gt=False, log='log_test.txt', mask=True, refine=False, resume='snapshot/checkpoint_e13.pth', save_mask=False, video='', visualization=False) [2020-02-02 19:55:54,343-rk0-features.py# 66] Current training 0 layers:

[2020-02-02 19:55:54,345-rk0-features.py# 66] Current training 1 layers:

[2020-02-02 19:55:54,355-rk0-features.py# 66] Current training 0 layers:

[2020-02-02 19:55:54,357-rk0-features.py# 66] Current training 1 layers:

[2020-02-02 19:55:54,361-rk0-features.py# 66] Current training 0 layers:

[2020-02-02 19:55:54,363-rk0-features.py# 66] Current training 1 layers:

Traceback (most recent call last): File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 608, in [2020-02-02 19:55:54,375-rk0-test.py#555] Namespace(arch='Custom', config='config.json', cpu=False, dataset='VOT2018', debug=False, gt=False, log='log_test.txt', mask=True, refine=False, resume='snapshot/checkpoint_e18.pth', save_mask=False, video='', visualization=False) main() File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 565, in main [2020-02-02 19:55:54,379-rk0-test.py#555] Namespace(arch='Custom', config='config.json', cpu=False, dataset='VOT2018', debug=False, gt=False, log='log_test.txt', mask=True, refine=False, resume='snapshot/checkpoint_e3.pth', save_mask=False, video='', visualization=False) assert isfile(args.resume), '{} is not a valid file'.format(args.resume) AssertionError: snapshot/checkpoint_e15.pth is not a valid file Traceback (most recent call last): File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 608, in main() File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 565, in main [2020-02-02 19:55:54,400-rk0-test.py#555] Namespace(arch='Custom', config='config.json', cpu=False, dataset='VOT2018', debug=False, gt=False, log='log_test.txt', mask=True, refine=False, resume='snapshot/checkpoint_e1.pth', save_mask=False, video='', visualization=False) assert isfile(args.resume), '{} is not a valid file'.format(args.resume) Traceback (most recent call last): AssertionError: snapshot/checkpoint_e7.pth is not a valid file File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 608, in main() File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 565, in main assert isfile(args.resume), '{} is not a valid file'.format(args.resume) AssertionError: snapshot/checkpoint_e20.pth is not a valid file [2020-02-02 19:55:54,410-rk0-test.py#555] Namespace(arch='Custom', config='config.json', cpu=False, dataset='VOT2018', debug=False, gt=False, log='log_test.txt', mask=True, refine=False, resume='snapshot/checkpoint_e14.pth', save_mask=False, video='', visualization=False) [2020-02-02 19:55:54,411-rk0-test.py#555] Namespace(arch='Custom', config='config.json', cpu=False, dataset='VOT2018', debug=False, gt=False, log='log_test.txt', mask=True, refine=False, resume='snapshot/checkpoint_e16.pth', save_mask=False, video='', visualization=False) [2020-02-02 19:55:54,413-rk0-test.py#555] Namespace(arch='Custom', config='config.json', cpu=False, dataset='VOT2018', debug=False, gt=False, log='log_test.txt', mask=True, refine=False, resume='snapshot/checkpoint_e4.pth', save_mask=False, video='', visualization=False) [2020-02-02 19:55:54,423-rk0-test.py#555] Namespace(arch='Custom', config='config.json', cpu=False, dataset='VOT2018', debug=False, gt=False, log='log_test.txt', mask=True, refine=False, resume='snapshot/checkpoint_e17.pth', save_mask=False, video='', visualization=False) [2020-02-02 19:55:54,437-rk0-test.py#555] Namespace(arch='Custom', config='config.json', cpu=False, dataset='VOT2018', debug=False, gt=False, log='log_test.txt', mask=True, refine=False, resume='snapshot/checkpoint_e9.pth', save_mask=False, video='', visualization=False) [2020-02-02 19:55:54,455-rk0-test.py#555] Namespace(arch='Custom', config='config.json', cpu=False, dataset='VOT2018', debug=False, gt=False, log='log_test.txt', mask=True, refine=False, resume='snapshot/checkpoint_e11.pth', save_mask=False, video='', visualization=False) [2020-02-02 19:55:54,457-rk0-test.py#555] Namespace(arch='Custom', config='config.json', cpu=False, dataset='VOT2018', debug=False, gt=False, log='log_test.txt', mask=True, refine=False, resume='snapshot/checkpoint_e19.pth', save_mask=False, video='', visualization=False) [2020-02-02 19:55:54,466-rk0-test.py#555] Namespace(arch='Custom', config='config.json', cpu=False, dataset='VOT2018', debug=False, gt=False, log='log_test.txt', mask=True, refine=False, resume='snapshot/checkpoint_e5.pth', save_mask=False, video='', visualization=False) [2020-02-02 19:55:54,573-rk0-features.py# 66] Current training 0 layers:

[2020-02-02 19:55:54,574-rk0-features.py# 66] Current training 1 layers:

[2020-02-02 19:55:54,589-rk0-features.py# 66] Current training 0 layers:

[2020-02-02 19:55:54,590-rk0-features.py# 66] Current training 1 layers:

[2020-02-02 19:55:54,603-rk0-features.py# 66] Current training 0 layers:

[2020-02-02 19:55:54,605-rk0-features.py# 66] Current training 1 layers:

Traceback (most recent call last): File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 608, in main() File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 565, in main assert isfile(args.resume), '{} is not a valid file'.format(args.resume) AssertionError: snapshot/checkpoint_e10.pth is not a valid file Traceback (most recent call last): File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 608, in [2020-02-02 19:55:54,631-rk0-features.py# 66] Current training 0 layers:

main()

File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 565, in main [2020-02-02 19:55:54,632-rk0-features.py# 66] Current training 1 layers:

assert isfile(args.resume), '{} is not a valid file'.format(args.resume)

AssertionError: snapshot/checkpoint_e8.pth is not a valid file [2020-02-02 19:55:54,637-rk0-features.py# 66] Current training 0 layers:

[2020-02-02 19:55:54,639-rk0-features.py# 66] Current training 1 layers:

Traceback (most recent call last): File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 608, in main() File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 565, in main assert isfile(args.resume), '{} is not a valid file'.format(args.resume) AssertionError: snapshot/checkpoint_e6.pth is not a valid file Traceback (most recent call last): File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 608, in main() File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 565, in main assert isfile(args.resume), '{} is not a valid file'.format(args.resume) AssertionError: snapshot/checkpoint_e12.pth is not a valid file [2020-02-02 19:55:54,669-rk0-features.py# 66] Current training 0 layers:

[2020-02-02 19:55:54,671-rk0-features.py# 66] Current training 1 layers:

[2020-02-02 19:55:54,674-rk0-features.py# 66] Current training 0 layers:

[2020-02-02 19:55:54,676-rk0-features.py# 66] Current training 1 layers:

Traceback (most recent call last): File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 608, in main() File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 565, in main assert isfile(args.resume), '{} is not a valid file'.format(args.resume) AssertionError: snapshot/checkpoint_e2.pth is not a valid file [2020-02-02 19:55:54,691-rk0-features.py# 66] Current training 0 layers:

[2020-02-02 19:55:54,692-rk0-features.py# 66] Current training 0 layers:

[2020-02-02 19:55:54,693-rk0-features.py# 66] Current training 1 layers:

[2020-02-02 19:55:54,693-rk0-features.py# 66] Current training 1 layers:

Traceback (most recent call last): File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 608, in main() File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 565, in main assert isfile(args.resume), '{} is not a valid file'.format(args.resume) AssertionError: snapshot/checkpoint_e4.pth is not a valid file Traceback (most recent call last): File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 608, in main() File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 565, in main assert isfile(args.resume), '{} is not a valid file'.format(args.resume) AssertionError: snapshot/checkpoint_e13.pth is not a valid file Traceback (most recent call last): File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 608, in main() File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 565, in main assert isfile(args.resume), '{} is not a valid file'.format(args.resume) AssertionError: snapshot/checkpoint_e17.pth is not a valid file [2020-02-02 19:55:54,728-rk0-features.py# 66] Current training 0 layers:

[2020-02-02 19:55:54,729-rk0-features.py# 66] Current training 0 layers: [2020-02-02 19:55:54,730-rk0-features.py# 66] Current training 1 layers:

[2020-02-02 19:55:54,731-rk0-features.py# 66] Current training 1 layers:

[2020-02-02 19:55:54,732-rk0-features.py# 66] Current training 0 layers:

[2020-02-02 19:55:54,733-rk0-features.py# 66] Current training 1 layers:

[2020-02-02 19:55:54,738-rk0-features.py# 66] Current training 0 layers:

Traceback (most recent call last): File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 608, in [2020-02-02 19:55:54,740-rk0-features.py# 66] Current training 1 layers:

main()

File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 565, in main assert isfile(args.resume), '{} is not a valid file'.format(args.resume) AssertionError: snapshot/checkpoint_e18.pth is not a valid file [2020-02-02 19:55:54,755-rk0-features.py# 66] Current training 0 layers:

[2020-02-02 19:55:54,757-rk0-features.py# 66] Current training 1 layers:

[2020-02-02 19:55:54,765-rk0-features.py# 66] Current training 0 layers: [2020-02-02 19:55:54,765-rk0-features.py# 66] Current training 0 layers:

Traceback (most recent call last): [2020-02-02 19:55:54,767-rk0-features.py# 66] Current training 1 layers: [2020-02-02 19:55:54,767-rk0-features.py# 66] Current training 1 layers: File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 608, in

main()

File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 565, in main assert isfile(args.resume), '{} is not a valid file'.format(args.resume) AssertionError: snapshot/checkpoint_e1.pth is not a valid file [2020-02-02 19:55:54,771-rk0-features.py# 66] Current training 0 layers:

[2020-02-02 19:55:54,772-rk0-features.py# 66] Current training 1 layers:

Traceback (most recent call last): File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 608, in Traceback (most recent call last): File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 608, in main() File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 565, in main main() assert isfile(args.resume), '{} is not a valid file'.format(args.resume) File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 565, in main AssertionError: snapshot/checkpoint_e3.pth is not a valid file assert isfile(args.resume), '{} is not a valid file'.format(args.resume) AssertionError: snapshot/checkpoint_e14.pth is not a valid file Traceback (most recent call last): File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 608, in main() File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 565, in main assert isfile(args.resume), '{} is not a valid file'.format(args.resume) AssertionError: snapshot/checkpoint_e16.pth is not a valid file Traceback (most recent call last): File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 608, in main() File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 565, in main assert isfile(args.resume), '{} is not a valid file'.format(args.resume) AssertionError: snapshot/checkpoint_e9.pth is not a valid file Traceback (most recent call last): File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 608, in main() Traceback (most recent call last): File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 565, in main File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 608, in Traceback (most recent call last): File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 608, in assert isfile(args.resume), '{} is not a valid file'.format(args.resume) AssertionError: snapshot/checkpoint_e19.pth is not a valid file main() File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 565, in main main() File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/test.py", line 565, in main assert isfile(args.resume), '{} is not a valid file'.format(args.resume) AssertionError: snapshot/checkpoint_e11.pth is not a valid file assert isfile(args.resume), '{} is not a valid file'.format(args.resume) AssertionError: snapshot/checkpoint_e5.pth is not a valid file Traceback (most recent call last): File "/media/nuist/0A9AD66165F33762/code/lxy/SiamMask/SiamMask/tools/eval.py", line 32, in assert len(trackers) > 0 AssertionError`

[cnctdadots]

The reason is you don't have snapshot/checkpoint_e9.pth inside SiamMask/experiments/siammask_base/. So if you are training the dataset from scratch, you can remove '--resume snapshot/checkpoint_e9.pth' part from run.sh.

[SamihaSara]

@tedrostime I faced this same error. Can you please explain "training the dataset from scratch"? I have downloaded and preprocessed the datasets according to readme.

[cnctdadots]

@SamihaSara, you can check inside the 'run.sh' file. it has --resume snapshot/checkpoint_e9.pth' as an option. but since you are starting to train it, you don't have checkpoint_e9.pth. for me what I did was remove that option, and it worked well.

[SamihaSara]

@tedrostime thanks for explaining this.

[humeranoor]

Hi there, I have the same error. And it doesn't work even if I remove the resume argument. It keeps on looking for snapshot/checkpoint_e1.pth, e2... e20 and fails, because none is generated. Did it work for you? Here is some partial output (the full would be too big)

[2020-09-23 18:15:43,597-rk0-features.py# 66] Current training 0 layers:
	
[2020-09-23 18:15:43,600-rk0-features.py# 66] Current training 1 layers:
	
[2020-09-23 18:15:43,606-rk0-features.py# 66] Current training 0 layers:
	
[2020-09-23 18:15:43,609-rk0-features.py# 66] Current training 1 layers:
	
[2020-09-23 18:15:43,614-rk0-features.py# 66] Current training 0 layers:
	
[2020-09-23 18:15:43,616-rk0-features.py# 66] Current training 0 layers:
	
[2020-09-23 18:15:43,617-rk0-features.py# 66] Current training 1 layers:
	[2020-09-23 18:15:43,618-rk0-features.py# 66] Current training 1 layers:
	
Traceback (most recent call last):
  File "/Users/humera/SiamMask/tools/test.py", line 608, in <module>
    main()
  File "/Users/humera/SiamMask/tools/test.py", line 565, in main
[2020-09-23 18:15:43,633-rk0-features.py# 66] Current training 1 layers:
	
    assert isfile(args.resume), '{} is not a valid file'.format(args.resume)
AssertionError: snapshot/checkpoint_e2.pth is not a valid file

[2020-09-23 18:15:43,643-rk0-features.py# 66] Current training 0 layers:
	[2020-09-23 18:15:43,644-rk0-features.py# 66] Current training 0 layers:
	
[2020-09-23 18:15:43,646-rk0-features.py# 66] Current training 1 layers:
	

[2020-09-23 18:15:43,659-rk0-features.py# 66] Current training 1 layers:
	
Traceback (most recent call last):
  File "/Users/humera/SiamMask/tools/test.py", line 608, in <module>
    main()
  File "/Users/humera/SiamMask/tools/test.py", line 565, in main
    assert isfile(args.resume), '{} is not a valid file'.format(args.resume)
AssertionError: snapshot/checkpoint_e3.pth is not a valid file
Traceback (most recent call last):
  File "/Users/humera/SiamMask/tools/test.py", line 608, in <module>
    main()
  File "/Users/humera/SiamMask/tools/test.py", line 565, in main
    assert isfile(args.resume), '{} is not a valid file'.format(args.resume)
AssertionError: snapshot/checkpoint_e14.pth is not a valid file`
```

[EEEEEREN]

Hi there, I have the same error. And it doesn't work even if I remove the resume argument. It keeps on looking for snapshot/checkpoint_e1.pth, e2... e20 and fails, because none is generated. Did it work for you? Here is some partial output (the full would be too big)

[2020-09-23 18:15:43,597-rk0-features.py# 66] Current training 0 layers:
	
[2020-09-23 18:15:43,600-rk0-features.py# 66] Current training 1 layers:
	
[2020-09-23 18:15:43,606-rk0-features.py# 66] Current training 0 layers:
	
[2020-09-23 18:15:43,609-rk0-features.py# 66] Current training 1 layers:
	
[2020-09-23 18:15:43,614-rk0-features.py# 66] Current training 0 layers:
	
[2020-09-23 18:15:43,616-rk0-features.py# 66] Current training 0 layers:
	
[2020-09-23 18:15:43,617-rk0-features.py# 66] Current training 1 layers:
	[2020-09-23 18:15:43,618-rk0-features.py# 66] Current training 1 layers:
	
Traceback (most recent call last):
  File "/Users/humera/SiamMask/tools/test.py", line 608, in <module>
    main()
  File "/Users/humera/SiamMask/tools/test.py", line 565, in main
[2020-09-23 18:15:43,633-rk0-features.py# 66] Current training 1 layers:
	
    assert isfile(args.resume), '{} is not a valid file'.format(args.resume)
AssertionError: snapshot/checkpoint_e2.pth is not a valid file

[2020-09-23 18:15:43,643-rk0-features.py# 66] Current training 0 layers:
	[2020-09-23 18:15:43,644-rk0-features.py# 66] Current training 0 layers:
	
[2020-09-23 18:15:43,646-rk0-features.py# 66] Current training 1 layers:
	

[2020-09-23 18:15:43,659-rk0-features.py# 66] Current training 1 layers:
	
Traceback (most recent call last):
  File "/Users/humera/SiamMask/tools/test.py", line 608, in <module>
    main()
  File "/Users/humera/SiamMask/tools/test.py", line 565, in main
    assert isfile(args.resume), '{} is not a valid file'.format(args.resume)
AssertionError: snapshot/checkpoint_e3.pth is not a valid file
Traceback (most recent call last):
  File "/Users/humera/SiamMask/tools/test.py", line 608, in <module>
    main()
  File "/Users/humera/SiamMask/tools/test.py", line 565, in main
    assert isfile(args.resume), '{} is not a valid file'.format(args.resume)
AssertionError: snapshot/checkpoint_e14.pth is not a valid file`

did u fix this question ? i removed "--resume snapshot/checkpoint_e9.pth", but it still report the question like u