BMCook

BMCook copied to clipboard

使用BMCook对GLM 10B模型压缩提速推理

1

你好，请问BMCook支持对GLM等大模型的模型压缩，提高推理速度吗？

jiangliqin

int8量化感知训练，保存的模型依然是fp32

3

通过BMCook进行模型压缩，配置了quantization和distillation，训练的loss收敛的很好。但是保存模型的时候，发现保存的checkpoint文件并没有减少，分析发现线性层的参数还是fp32的。 另外bmtrain的优化器AdamOffloadOptimizer和AdamOptimizer也只支持参数保存为fp32和fp16，并没有实现参数保存为int8。

jinmin527

请问支持的模型是和BMTrain支持的模型列表一样的吗？有没有支持MOSS、chatGLM2-6B，RWKV？

iclgg

Traceback (most recent call last): File "gpt2_test.py", line 186, in Traceback (most recent call last): File "gpt2_test.py", line 186, in main() File "gpt2_test.py", line 142, in main if outputs.lag_loss !=...

AlexanderChen1989

ImportError: /home/miniconda3/envs/BMCook/lib/python3.10/site-packages/bmtrain/nccl/_C.cpython-310-x86_64-linux-gnu.so: undefined symbol: ncclBroadcast ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 17198) of binary: /home/miniconda3/envs/BMCook/bin/python

3

Hi, I encountered the error described in the title of this issue, while trying to run the gpt-2 example. Here is my command: ``` export CUDA_VISIBLE_DEVICES=7 torchrun --nnodes=1 --nproc_per_node=1 --rdzv_id=1...

wln20

TypeErrorTypeError: : AdamOptimizer.__init__() got an unexpected keyword argument 'scale'AdamOptimizer.__init__() got an unexpected keywo$ d argument 'scale'

1

When I try to run the examples :`bash gpt/gpt2_test.sh` It fails and throws out the following errors: > File "/workspace/BMCook/examples/gpt/gpt2_test.py", line 84, in main File "/workspace/BMCook/examples/gpt/gpt2_test.py", line 84, in main...

Oran-Ac