Cyril Vallez comments

Results 63 comments of


                                            Cyril Vallez

Reducing memory usage: removing useless logits computation in generate()

Hi @ringohoffman, don't worry I am not forgetting about this 😉 I'm currently on vacation so I will try to wrap it up quickly end of August when I come...

Reducing memory usage: removing useless logits computation in generate()

Gemma2 was not released yet when I started this, but don't worry I will add it as well, it's on the roadmap 🤗

Reducing memory usage: removing useless logits computation in generate()

@ArthurZucker I added support for Gemma2 as well as tests, ready for last review 🤗 Red CIs are not related to the PR

Reducing memory usage: removing useless logits computation in generate()

My bad, there was actually a check-copy inconsistency that was my doing. Fixed it! Last red CI is the `libtorch_cuda.so: cannot open shared object file: No such file or directory`...

Reducing memory usage: removing useless logits computation in generate()

Indeed it worked, @ArthurZucker everything is now green!

Reducing memory usage: removing useless logits computation in generate()

No worries! All good on the CIs and ready to be merged 🤗

Fix bugs in DynamicCache

We just need to fix the small conflict based on our new ruff rules, then it's good to go!

PerceptionLM

run-slow: perception_lm

PerceptionLM

run-slow: perception_lm

Add Aria

The battle continues ⚔️! Looking into the modular issue