JaonLiu

Results 72 comments of JaonLiu

Add inference code

> No, I didn't encounter that error. Can you give me more context? just use : ``` instructions = [ "模仿鲁迅的风格, 吐槽一下最近食堂饭菜涨价", ] ```

GPTQ-int4 vllm部署出错

Qwen1.5-0.5B-Chat-GPTQ-Int4、Qwen1.5-0.5B-Chat-GPTQ-Int8、Qwen1.5-0.5B-Chat-GPTQ-Int8 这3个模型用vllm部署后，请求都报错了。只有Qwen1.5-0.5B-Chat-AWQ和Qwen1.5-0.5B-Chat可以正常返回。 报错内容都是： ``` ransformers/tokenization_utils_fast.py", line 612, in convert_tokens_to_string return self.backend_tokenizer.decoder.decode(tokens) TypeError: argument 'tokens': 'NoneType' object cannot be converted to 'PyString' ```

网页问答异常

@JianxinMa 求助

ERROR: Could not build wheels for xformers, which is required to install pyproject.toml-based projects

same problem，+1008611

Can not train llama-7b due to OOM on 40GA100

same question

Can not train llama-7b due to OOM on 40GA100

> > same OOM question > > Set placement_policy='cpu' can alleviate this question. How long will it cost to run train_sft.py only use CPU？

Can not train llama-7b due to OOM on 40GA100

Heal our children!

Can not train llama-7b due to OOM on 40GA100

> we run our LLama 7B 4 * A100 80G, if you want to run it on 40 G A100, you can use a smaller batch size and expand accimulation_steps...

Does Post-training full integer quantization support BERT?

@yyoon Could you help to solve it? Thanks a lot!

载入数据问题：load_msra_ner_without_dev

都没看到 load_msra_ner_without_dev 这个函数在哪里定义的