IF Lighter text encoder

Lighter text encoder

Open igor-davidyuk opened this issue 2 years ago • 2 comments

Hey guys, thank you for the awesome repo! Is there any chance that you will support a lighter tex encoder?

May 08 '23 12:05 igor-davidyuk

If by lighter you mean requiring less VRAM, then you can try to set the dtype to float16. T5 is then about 11.6GiB.

May 09 '23 16:05 phalexo

If by lighter you mean requiring less VRAM, then you can try to set the dtype to float16. T5 is then about 11.6GiB.

Thank you for your advice, but my CU does not support fp16 operations. Anyway, it looks like T5 XXL is too big and a smaller embedder would do

May 10 '23 10:05 igor-davidyuk