swift

swift copied to clipboard

我想将模型保存到本地，怎么才能保存。同时怎么调用本地的模型？

AlexJJJChen

internvl-chat-v1.5-int8 推理时报错，应该如何处理

3

**Describe the bug** What the bug is, and how to reproduce, better with screenshots(描述bug以及复现过程，最好有截图) 可以正常启动 CUDA_VISIBLE_DEVICES=0 swift infer --model_type internvl-chat-v1_5-int8 --model_id_or_path /home/tione/notebook/community/scan/InternVL-Chat-V1-5-int8/ --dtype bf16 但是推理时报错 internvl-chat-v1_5可以正常启动和推理 CUDA_VISIBLE_DEVICES=0 swift infer --model_type...

wlg-tt

可以支持一下RoPE外推不

1

对于比较长的长下文微调帮助挺大的

zhangfan-algo

**Describe the bug** 2024-05-16 14:19:20 [W socket.cpp:697] [c10d] The IPv6 network addresses of (zf-yi1-5-34b-sft-0516-02-master-0, 23456) cannot be retrieved (gai error: -2 - Name or service not known). 2024-05-16 14:19:35 Traceback...

zhangfan-algo

训练qwen14b，前面lr一直为0

1

tried to get lr value before scheduler/optimizer started stepping, returning lr=0

jhjiang10

**Describe the feature** TensorRT10发布了，同时还有TensorRT-LLM，是否能用其对训练和推理加速呢？ **Paste any useful information** 下述来自NVIDIA的推广邮件 The TensorRT ecosystem of API releases include TensorRT 10.0, TensorRT-LLM 0.10, and TensorRT Model Optimizer 0.11. Highlights from this release include:...

WSC741606

量化后的模型推理报错怎么解决

1

 **Describe the bug** What the bug is, and how to reproduce, better with screenshots(描述bug以及复现过程，最好有截图) 量化后的模型推理报错 CUDA_VISIBLE_DEVICES=0 swift infer --model_type baichuan2-7b --model_id_or_path baichuan2-7b-gptq-int4 **Your hardware and system info** Write your...

greatheart1000

多模态微调支持多轮对话吗

1

看到文档里面多模态的最佳实践，上面写着： (只支持单轮对话, 每轮对话必须包含一张图片, 支持传入本地路径或URL) 后续可以支持多轮对话吗

LRHstudy

deepseek-vl-7b模型使用deepspeed的ZeRo3报错

1

执行--deepspeed default-zero2是可以正常运行，在尝试使用--deepspeed default-zero3时报错，具体执行命令如下： NPROC_PER_NODE=2 CUDA_VISIBLE_DEVICES=1,2 swift sft --model_type deepseek-vl-7b-chat --custom_train_dataset_path data/train.json --custom_val_dataset_path data/val.json --sft_type full --eval_steps 100 --deepspeed default-zero3 报错信息如下： result = llm_x(args, **kwargs) File "/mnt/MLLM/swift/swift/llm/sft.py", line 179, in llm_sft...

jiujiuma

用qwen-7b-int4和int8进行lora微调后，微调和推理没问题，但部署后，请求报错

1

openai方式请求报错  **Describe the bug** INFO: 127.0.0.1:35572 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error ERROR: Exception in ASGI application Traceback (most recent call last): File "/home/chaiy/software/miniconda3/envs/swift/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 411, in...

nauyiahc