BELLE
BELLE copied to clipboard
synchronize before creating output_dir
In multiprocessing, log file may be created before other processes checking if len(os.listdir(training_args.output_dir)) > 0
, and thus a ValueError
will be raised.
Synchronizing between processes using torch.distributed.barrier()
tackles this problem.