Unintensored comments

Repositories
Issues
Comments

Results 1 comments of


                                            Unintensored

Authors, any suggestion on why inference speed is slow

I think it has to do with the usage of 8-bit for LLaMA text generation. By default, MiniGPT-4 now uses bitsandbyte's load_in_8bit to reduce the VRAM footprint. However, this results,...