Raushan Turganbay

Hey @kirk86 ! Thanks for pointing out this issue. We are in the process of making generation more generalizable and easy to integrate with different model types. But in the...

@antonioalegria In the case of this code snippet the model is loaded in `float32`, and I believe running a trainer.train() does not change the model itself to fp16 dtype. Can...

@edchengg I could localize the error. When using trainer it has a line that prepares model with `accelerate`, which in turn adds something like `model.forward = convert_outputs_to_fp32(new_forward)` casting all model...

1. Cool, I did not notice that 2 and 3. Okay, thought we need it for consistency like we had with other new args in public classes. Will remove it...

Not stale

This PR now can be reviewed. Rebased main and updated the changes. All the tests from `RUN_SLOW=1 pytest tests/generation` are passing on my end

@LinB203 hey! As we talked before, here is a draft PR of Video Llava. I checked that the modeling part runs without errors and generates similar to the original repo....

@LinB203 pinging in case the first one got lost in notifications :)

I believe we can start reviewing this now. I converted weights and added them to my hub account temporarily, so that we can run and test the model. In the...

Rebased with main and resolved conflicts. The only failing doctest seems to be not able to load and run 7b model in 120sec, but I think we will leave it...