add gaudi modeling support in itrex

[ClarkChin08]

Type of Change

gaudi modeling used in itrex for int4 kv-cache support

[github-actions[bot]]

⚡ Required checks status: All passing 🟢

Groups summary

🟢 Format Scan Tests workflow

Check ID	Status	Error details
format-scan (pylint)	success		✅
format-scan (bandit)	success		✅
format-scan (cloc)	success		✅
format-scan (cpplint)	success		✅

These checks are required after the changes to intel_extension_for_transformers/neural_chat/examples/finetuning/multi_modal/eval/mmmu_eval/run_llava.py, intel_extension_for_transformers/neural_chat/models/model_utils.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/generation/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/generation/configuration_utils.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/generation/stopping_criteria.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/generation/utils.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/modeling_utils.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/albert/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/albert/modeling_albert.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/bart/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/bart/modeling_bart.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/blip/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/blip/modeling_blip.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/blip/modeling_blip_text.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/bloom/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/bloom/modeling_bloom.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/codegen/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/codegen/modeling_codegen.py and 39 more files...

🟢 Optimize Unit Test workflow

Check ID	Status	Error details
optimize-unit-test-baseline	success		✅
optimize-unit-test-PR-test	success		✅
Genreate-OptimizeUT-Report	success		✅

These checks are required after the changes to intel_extension_for_transformers/transformers/modeling/modeling_gaudi/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/generation/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/generation/configuration_utils.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/generation/stopping_criteria.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/generation/utils.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/modeling_utils.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/albert/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/albert/modeling_albert.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/bart/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/bart/modeling_bart.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/blip/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/blip/modeling_blip.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/blip/modeling_blip_text.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/bloom/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/bloom/modeling_bloom.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/codegen/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/codegen/modeling_codegen.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/esm/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/esm/modeling_esmfold.py and 36 more files...

🟢 NeuralChat Unit Test

Check ID	Status	Error details
neuralchat-unit-test-baseline	success		✅
neuralchat-unit-test-PR-test	success		✅
Generate-NeuralChat-Report	success		✅

These checks are required after the changes to intel_extension_for_transformers/neural_chat/models/model_utils.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/generation/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/generation/configuration_utils.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/generation/stopping_criteria.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/generation/utils.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/modeling_utils.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/albert/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/albert/modeling_albert.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/bart/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/bart/modeling_bart.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/blip/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/blip/modeling_blip.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/blip/modeling_blip_text.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/bloom/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/bloom/modeling_bloom.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/codegen/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/codegen/modeling_codegen.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/esm/__init__.py and 37 more files...

🟢 Engine Unit Test workflow

Check ID	Status	Error details
engine-unit-test-baseline	success		✅
engine-unit-test-PR-test	success		✅
Genreate-Engine-Report	success		✅

These checks are required after the changes to intel_extension_for_transformers/transformers/modeling/modeling_gaudi/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/generation/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/generation/configuration_utils.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/generation/stopping_criteria.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/generation/utils.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/modeling_utils.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/albert/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/albert/modeling_albert.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/bart/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/bart/modeling_bart.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/blip/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/blip/modeling_blip.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/blip/modeling_blip_text.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/bloom/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/bloom/modeling_bloom.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/codegen/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/codegen/modeling_codegen.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/esm/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/esm/modeling_esmfold.py and 36 more files...

🟢 Chat Bot Test workflow

Check ID	Status	Error details
call-inference-llama-2-7b-chat-hf / inference test	success		✅
call-inference-mpt-7b-chat / inference test	success		✅

These checks are required after the changes to intel_extension_for_transformers/neural_chat/models/model_utils.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/generation/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/generation/configuration_utils.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/generation/stopping_criteria.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/generation/utils.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/modeling_utils.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/albert/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/albert/modeling_albert.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/bart/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/bart/modeling_bart.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/blip/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/blip/modeling_blip.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/blip/modeling_blip_text.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/bloom/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/bloom/modeling_bloom.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/codegen/__init__.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/codegen/modeling_codegen.py, intel_extension_for_transformers/transformers/modeling/modeling_gaudi/models/esm/__init__.py and 37 more files...

---

Thank you for your contribution! 💜

Note This comment is automatically generated and will be updates every 180 seconds within the next 6 hours. If you have any other questions, contact VincyZhang or XuehaoSun for help.

[ClarkChin08]

@lkk12014402 kaokao, please take a look on the new commit on measurement of ppl