regisss

@mandy-li I believe the RoPE implementation changed because of https://habana.atlassian.net/servicedesk/customer/portal/1/HS-1574. Can you confirm that you reach a perplexity of about 8 or 9 with this PR when you run ```...

> @regisss , good point. i didn't know this jira when i worked on the type casting. The reason why we have this PR is because our QA reported perf...

@anishagartia It's probably an error in the tokenizer config of `bigcode/starcoder2-15b-instruct-v0.1`. Let me check that internally and I'll follow up with you.

@anishagartia The issue should be solved now, can you try again and let me know if it works on your side?

@sywangyi I just merged #809, can you merge it into this branch and solve the merge conflicts please?

@sywangyi Why calling the new arg `--no_ignore_eos` and not simply `--ignore_eos` as everywhere else in the codebase? If you just want to have it true by default, I think this...

I pushed a new commit where the logic is simpler. It only works with Python 3.9+, which is fine as Habana Docker images don't support Ubuntu 20.04 anymore.

@skavulya When running `make style`, `ruff` found 2 errors: ``` optimum/habana/transformers/generation/utils.py:1958:24: F821 Undefined name `prev_idx` | 1956 | if model_kwargs.get("token_idx_cpu")

> @regisss I saw the PR for upgrade to 4.43 #1163 Would you like me to upgrade this PR too? That would be great if you have time to do...

@MLikeWater What do you mean exactly by adaptive output?