使用CUDA_VISIBLE_DEVICES环境变量和bloom_inference.py 无法实现双卡推理

[blizzardwj]

执行shell 命令： CUDA_VISIBLE_DEVICES=0,1 python bloom_inference.py BELLE-7B-2M --text "hello" 提示： torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 256.00 MiB (GPU 0; 23.69 GiB total capacity; 22.83 GiB already allocated; 162.94 MiB free; 22.83 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF