Jeonghoon

icon company NAVER icon location Seoul, Korea icon location Quantization of Large Language Model

Results 1 comments of Jeonghoon

[BUG] Deepspeed Zero 3 Inference InFlight Params with new HuggingFace Mixtral Model

@ftgreat Hello, I would like to know if your monkey patch can achieve the same results as the original mixtral forward. Is this method currently the best approach right?