请问什么时候可以支持INT4推理？

[ThisisBillhe]

在rk3588这一代的npu介绍里是有INT4的支持，请问现在是否有接口使用？或者未来什么时候会有支持？

[keveman]

I have had good performance with fp16 for transformer models, especially OpenAI's Whisper, using the rknn_matmul API. Have a look here : https://github.com/usefulsensors/useful-transformers int4 would really make it much more performant for other LLM transformer models. cc: [email protected]