Wei-Hsin Chen
Results
1
comments of
Wei-Hsin Chen
** when we using fp8 / int4_awq with not enough vram. we saw it would automatic offload some parameters. it's really thankful! however, it would have some bug to save...