houliangxue

Results 4 comments of houliangxue

qwen1.5-7b-chat模型4090内存溢出

> torch_dtype 设置成bfloat16就ok了 qwem1.5的config.json默认就是bfloat16呀，提示词比较长的时候还是会爆显存

单机单卡4090，lora微调deepseek-r1-distill-1.5B卡死，稳定复现，会导致整个机器卡死，nvdia-smi命令失效，

![Image](https://github.com/user-attachments/assets/0dbe828c-fe50-407b-8723-c023060b1055)

Fun-CosyVoice3-0.5B的效果简直就是噩梦

提示词与2不一样哦 ，需要'You are a helpful assistant.’

非流式比流式更快吗

import sys import markdown from bs4 import BeautifulSoup from fastapi import FastAPI, HTTPException from fastapi.responses import StreamingResponse, Response import uvicorn from pydantic import BaseModel import torchaudio import io import os...