[Bug] [model] vllm run error

[mdjhacker]

Search before asking

• [X] I had searched in the issues and found no similar issues.

Operating system information

Linux

Python version information

=3.11

DB-GPT version

main

Related scenes

• [ ] Chat Data
• [ ] Chat Excel
• [ ] Chat DB
• [X] Chat Knowledge
• [ ] Model Management
• [ ] Dashboard
• [ ] Plugins

Installation Information

• [ ] Installation From Source

• [ ] Docker Installation

• [ ] Docker Compose Installation

• [ ] Cluster Installation

• [ ] AutoDL Image

• [ ] Other

Device information

有足够的内存与显存

Models information

LLM_MODEL=chatglm2-6b

What happened

在使用chatglm2进行运行程序的时候，vlllm报错： dbgpt.model.cluster.worker.default_worker[575445] ERROR Model inference error, detail: Traceback (most recent call last): File "/mnt/c/Users/DELL/PycharmProjects/DB-GPT/dbgpt/model/cluster/worker/default_worker.py", line 238, in async_generate_stream async for output in generate_stream_func( File "/mnt/c/Users/DELL/PycharmProjects/DB-GPT/dbgpt/model/llm_out/vllm_llm.py", line 26, in generate_stream if tokenizer.eos_token_id is not None: AttributeError: 'NoneType' object has no attribute 'eos_token_id'

What you expected to happen

1

How to reproduce

1

Additional context

1

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

[github-actions[bot]]

This issue has been marked as stale, because it has been over 30 days without any activity.

[likenamehaojie]

ME TO

[likenamehaojie]

i have the same problem, i use codellma-13b

[programmermw1986]

same problem

[xiyuanyuan0506]

i have the same problem, i use codellma-13b 请问这个问题你解决了吗

[Xavier-123]

请问有人解决了吗

[github-actions[bot]]

This issue has been marked as stale, because it has been over 30 days without any activity.

[github-actions[bot]]

This issue bas been closed, because it has been marked as stale and there has been no activity for over 7 days.