onediff icon indicating copy to clipboard operation
onediff copied to clipboard
verified publisher icon siliconflow

[Feature Request] fp8 support

Open ShirasawaSama opened this issue 2 years ago • 2 comments

If the model can be loaded in FP8, it seems to significantly reduce the vram requirements. SDXL only requires 3g of vram to run.

https://github.com/comfyanonymous/ComfyUI/blob/18c151b3e3f6838fab4028e7a8ba526e30e610d3/comfy/model_management.py#L495

ShirasawaSama avatar Feb 21 '24 06:02 ShirasawaSama

fp8 is not supported for the moment. But it's on the roadmap(May take some months in our new version).

strint avatar Feb 21 '24 07:02 strint

Ok, thank you very much.

ShirasawaSama avatar Feb 21 '24 07:02 ShirasawaSama