About multi cards inference.

[HandsomeWuuu]

the code of "inference/infer_single.py" can not infer "TRACE-Benchmark/LLM-CL-Benchmark_1000", due to CUDA OOM(A100 8G) and only can work on the single card. it's not reasonable.

I think the code of "inference/infer_multi.py" maybe for multi-card, but it have some bug for infer lora, after i fixed it, the infer results of every task is ""(nothing). maybe the "def prediction(model, infer_dataloader):" have some trouble.

can you check it? thanks for your work and help.