FlexGen icon indicating copy to clipboard operation
FlexGen copied to clipboard

Killed Issue with flexgen when running python script

Open foreverpiano opened this issue 10 months ago • 2 comments

python3 -m flexgen.flex_opt --model facebook/opt-30b --percent 0 100 100 0 100 0 --offload-dir /scratch/bcjw/ding3/flexgen_offload_dir  --path /scratch/bcjw/ding3/opt_weights
<run_flexgen>: args.model: facebook/opt-30b
model size: 55.803 GB, cache size: 2.789 GB, hidden size (prefill): 0.029 GB
init weight...
Killed

I use this script. it shows killed and has no output.

foreverpiano avatar Apr 15 '24 03:04 foreverpiano

run on 2xA100 node, pytorch 2.2.1 @Ying1123 @BinhangYuan

foreverpiano avatar Apr 15 '24 03:04 foreverpiano

it seems stuck at self.init_weight(j) where j = 68 for opt-30b

foreverpiano avatar Apr 18 '24 08:04 foreverpiano