Baichuan-13B Model failed to load on HYGON DCU.

Model failed to load on HYGON DCU.

Open baichuan-assistant opened this issue 1 year ago • 4 comments

During inference, when loading the model, it gets KILLED when 1/3 of the files (total of 3 files) are loaded, with no other log records.

Jul 19 '23 04:07 baichuan-assistant

Also it seems HYGON DCU can support LLAMA1 and GLM.

Jul 19 '23 05:07 baichuan-assistant

Maybe your memory is not enough

Jul 22 '23 10:07 jameswu2014

Had 8 nodes, each with 4 * 32G vRAM

CPU and memory:

Jul 24 '23 08:07 baichuan-assistant