xiaoyuer2019 comments

Results 9 comments of


                                            xiaoyuer2019

开源的模型里哪个是100K上下文的版本？

我这里只有9块3090的显卡，请问有v6量化版的么？量化版的是不是可以支持100K？

量化模型使用下面参数启动模型 export PYTHONPATH='./' ; CUDA_VISIBLE_DEVICES=1,2,3,4 streamlit run apps/exllamav2_web_demo.py -- --model_path /data/model/tigerbot-70b-chat-v6-4bit-exl2/tigerbot --max_input_length 37888 --max_generate_length 62112 在长文本推理的时候会报下面的错误 Truncation was not explicitly activated but `max_length` is provided a specific value, please use...

开源的模型里哪个是100K上下文的版本？

13B模型使用下面参数启动模型 export PYTHONPATH='./' ; export CUDA_VISIBLE_DEVICES=1,2,3,4,5,6,7,8 ; streamlit run apps/web_demo.py -- --model_path /data/model/tigerbot-13b-chat-v6 --rope_scaling yarn --rope_factor 8 --max_input_length 10240 --max_generate_length 10240 在长文本推理的时候会报下面的错误 Namespace(model_path='/data/model/tigerbot-13b-chat-v6', rope_scaling='yarn', rope_factor=8.0, max_input_length=10240, max_generate_length=10240) Truncation was not...

[BUG] <title>启动embedding模型报错

在docker里运行也是报这个错误 /workspace/QAnything UPLOAD_ROOT_PATH: /workspace/QAnything/QANY_DB/content IMAGES_ROOT_PATH: /workspace/QAnything/qanything_kernel/qanything_server/dist/qanything/assets/file_images /usr/local/lib/python3.10/site-packages/transformers/utils/generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. _torch_pytree._register_pytree_node( args: Namespace(use_gpu=False, workers=1) [2025-03-05 06:15:40 +0000] [15] [INFO] Sanic v23.6.0 [2025-03-05 06:15:40 +0000] [15]...