[Bug] LLMServer Generate Error, Please CheckErrorInfo.: CUDA error

[langmansh]

Search before asking

• [X] I had searched in the issues and found no similar issues.

Operating system information

Windows

Python version information

3.10

DB-GPT version

main

Related scenes

• [x] Chat Data
• [ ] Chat Excel
• [ ] Chat DB
• [ ] Chat Knowledge
• [ ] Model Management
• [ ] Dashboard
• [ ] Plugins

Installation Information

• [ ] Installation From Source

• [ ] Docker Installation

• [ ] Docker Compose Installation

• [ ] Cluster Installation

• [ ] AutoDL Image

• [ ] Other

Device information

CPU and GPU

Models information

chatglm3-6b bge-large-zh-v1.5

What happened

当我向chat data提问的时候，使用cuda的时候报错：ERROR!Model server error!code=1, error msg is LLMServer Generate Error, Please CheckErrorInfo.: CUDA error: device-side assert triggered CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

使用cpu的时候报错：ERROR!Model server error!code=1, error msg is LLMServer Generate Error, Please CheckErrorInfo.: index 26602 is out of bounds for dimension 0 with size 8192

What you expected to happen

请问这两个问题如何解决的？

How to reproduce

请问这两个问题如何解决的？

Additional context

No response

Are you willing to submit PR?

• [X] Yes I am willing to submit a PR!

[github-actions[bot]]

This issue has been marked as stale, because it has been over 30 days without any activity.

[github-actions[bot]]

This issue bas been closed, because it has been marked as stale and there has been no activity for over 7 days.

[zhangkuo-zk]

我本地部署的chatglm3-6b模型也遇见这样的错误