SafeCool issues

Repositories
Issues
Comments

Results 2 issues of


                                            SafeCool

user trtllm-serve error

### System Info RuntimeError: Failed to import transformers.models.bert.modeling_bert because of the following error (look up to see its traceback): /usr/local/lib/python3.12/dist-packages/flash_attn_2_cuda.cpython-312-x86_64-linux-gnu.so: undefined symbol: _ZN3c105ErrorC2ENS_14SourceLocationENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE ### Who can help? _No response_ ###...

bug

need more info

The LLM I deployed myself, such as Qwen, is in the form of an openai interface. How should I evaluate the effectiveness of this model? Why didn't I find it in the documentation and can only use GPT?