ming-shy

Isn't vllm compatible with other web frameworks?

8

**_Dear author, At the bottom I used the flask framework to build the web service, but there were some strange phenomena (below). Isn't vllm compatible with other web frameworks?_** **phenomena**...

ADD a large model SSE stream api

2

…ng interface **What this PR does / why we need it**: _**Due to business needs, I added a large model SSE streaming interface to the seldon-core microservice component. But instead...

如何使用FasterTransformer适配自己的模型

1

Dear author, hello: I am very interested in the architecture of FasterTransformer. I have been working as an AI algorithm engineer for a year, and I can transfomer effectively. It...

想请教一下作者，如何让FasterTransformer适配自己的模型，需要怎么学习，现在似乎没有一些特定的教程，不知道从何学起。请赐教一下，非常感谢。

[BUG] <title>Qwen2-7b-instruct使用SFT-FT，loss变为0，如何解决？

1

### 是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this? - [X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions ### 该问题是否在FAQ中有解答？ | Is there an...

Qwen2-7b-instruct使用SFT-FT，loss变为0，如何解决？

4

我的训练参数如下： export CUDA_VISIBLE_DEVICES="1,6,7" && DS_SKIP_CUDA_CHECK=1 OMP_NUM_THREADS=8 torchrun --standalone --nproc_per_node=gpu train_qwen.py \ --model_name_or_path $model_path \ --deepspeed $deepspeed_config \ --data_path $data_path \ --bf16 True \ --output_dir $output_path \ --num_train_epochs 1 \ --max_steps...