LongBench
LongBench copied to clipboard
No initialization for the process group
In pred.py, there's dist.destroy_process_group() without any initialization before. So it throws an assertion error.
Hi, you can change this line to
if dist.is_initialized():
dist.destroy_process_group()
to avoid the assertion error.