Zhengfu He

Mechanistic Interpretability Research @OpenMOSS Team @FudanNLP.

Results 19 comments of


Zhengfu He

Missing key(s) in state_dict when testing using predict_downstream_condition.py

Hi, Did you train the model with DDP? If so, the state dict keys may be different.

Add auto-gptq integration

感谢您的PR. 看了一下autogptq的安装，默认会重装torch和cuda ext。这对于多数用户来说感觉不够友好，能否为MOSS设计一个pip install 的最小依赖集合，可以在现有的环境上便捷地安装？

NameError: name 'autotune' is not defined

已更新

经过几番捣腾，后台报语法错误：TypeError: '<' not supported between instances of 'tuple' and 'float'`

请查看这个issue https://github.com/OpenLMLab/MOSS/issues/65

经过几番捣腾，后台报语法错误：TypeError: '<' not supported between instances of 'tuple' and 'float'`

已更新

运行报错

这是个warning，应该不会影响实际效果

使用moss-moon-003-sft-int8量化模型后会报错 is not a folder containing a `.index.json`

尝试这样更新主函数 model = MossForCausalLM.from_pretrained("fnlp/moss-moon-003-sft-int4").half().cuda() infer = Inference(model, device_map="auto")

使用moss-moon-003-sft-int8量化模型后会报错 is not a folder containing a `.index.json`

量化算法对内存要求较高

Triton cannot be installed

We are currently working on it. The workaround may be slower.

fnlp/moss-moon-003-sft-int4 量化版本报错

报错的triton版本是什么呢？