ncnn icon indicating copy to clipboard operation
ncnn copied to clipboard

INT8 量化以后没法用GPU推理

Open dimension1234 opened this issue 9 months ago • 2 comments

ncnn INT8量化以后无法用GPU yolo11.opt.use_vulkan_compute = true; //yolo11.opt.use_int8_inference = true; //yolo11.opt.use_int8_storage = true; //yolo11.opt.use_bf16_storage = true;

//yolo11.load_param("yolo11/yolov8n.ncnn.param"); //yolo11.load_model("yolo11/yolov8n.ncnn.bin"); yolo11.load_param("yolo11/yolov8n-int8.param"); yolo11.load_model("yolo11/yolov8n-int8.bin");
请教大神,我将yolov8n进行INT8量化以后,进行推理,原始模型实际观察GPU状态是启用了的,速度也比CPU快将近一倍; 但是INT8模型一运行,倒是也能出来结果,但是GPU占用是0,说明并没有用GPU进行推理,而且速度确实慢了。 是我量化过程有问题吗?

dimension1234 avatar May 13 '25 09:05 dimension1234

ncnn没有int8的vulkan shader #5996

futz12 avatar May 30 '25 04:05 futz12