Any example to quantise a text embedding model on Intel Gaudi2?

[sleepingcat4]

I was looking for example or documentation how I can load or quantise both a HF embedding model on Intel Gaudi2. is there any examples available? I don't want to use docker btw

[NeoZhangJianyu]

@sleepingcat4 Please refer to: https://github.com/intel/neural-compressor/tree/bfa27e422dc4760f6a9b1783eee7dae10fe5324f/examples/3.x_api/pytorch/nlp/huggingface_models/language-modeling/quantization/habana_fp8.

[sleepingcat4]

thank you! I will experiment with it tomorrow

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 7 days with no activity.