regisss

@xin3he When I run ``` GAUDI2_CI=1 pytest tests/test_text_generation_example.py -v -s -k "mistralai/Mistral-7B-v0.1" --token my_token ``` I get the following error: ``` File "/usr/local/lib/python3.10/dist-packages/optimum/habana/transformers/models/mistral/modeling_mistral.py", line 67, in update assert cur.shape[2] ==...

Upgraded to Transformers v4.40 in #1027.

@tkanvar Can you update the main branch of your fork and merge it into your branch please? The current state of the PR displays many unrelated changes that were already...

That's already the case, Accelerate is pinned here: https://github.com/huggingface/optimum-habana/blob/060ddee48f247f049973cde02b2bb3b055febf52/setup.py#L35 If you have Accelerate v0.28, there is an issue in the way Optimum Habana was installed. Can you try again in...

There are also some merge conflicts to solve now after the 2 PRs I just merged

Yes this implementation is quite experimental as it was mainly for testing a few things. I'll push a cleaner one soon.

@dsocek LGTM! Could you also add a test in `tests/test_diffusers.py` to test the throughput and that the example works please?

I'm going to look into it

> @regisss - Did you get some time to check this behaviour Not yet. I don't think I'll have time to do it this week and before releasing Optimum Habana...

Hi @Denny-kef! I assume you're talking about adding support for ONNX export here. I could see this [for loop depending on the batch size](https://github.com/huggingface/transformers/blob/9728f1134bf1bcb822dcb46a205e19f12ff21323/src/transformers/models/mask2former/modeling_mask2former.py#L465) messing up with dynamic shapes. But...