intel-extension-for-transformers

intel-extension-for-transformers copied to clipboard

None of examples on README page works

5

Same errors on 3 different Linux distros. I have installed from source: pushd intel-extension-for-transformers/ pip install -r requirements.txt python setup.py install Then start to try examples from README (obviously, my...

olegmikul

Deploying on virtual machines?

1

Anybody knows if there are issues running this on Virtual Machines? I'm testing this on a homelab with Proxmox running on relatively old equipment (dual Xeon E5-2690v4), and I can't...

fidecastro

Why is the inference time so long on CPU(i9-13900) of Win11?

33

I used intel-extension-for-transformers of INT4 LLaMA2 model. The model is from python scripts/covert.py and python scripts/quantize.py. The operation system is Win11, and the cpu is 13th Gen Intel(R) Core(TM) i9-13900HX....

zhengshuo1

Inference ONNX format model

5

Hi guys, Are you planning to inference onnx format model soon?

bmtuan

environment problem in qat for stable diffusion

1

I followed ``` pip install intel-extension-for-transformers pip install -r requirements.txt pip install transformers==4.34.1 ``` but there exists hugging-face version conflict, and I checked dependabot's advice(#981) upgrade transformers version to 4.36.0....

Sugar929

Feature/support older intel mac book pro with gcc 13

1

## Type of Change Initiated discussion https://github.com/intel/intel-extension-for-transformers/discussions/1081 to while trying to get this project to compile on a 2019 Intel-based MacBook Pro. Currently segfaults when running inference on [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) quantized...

nezda

An error occurred during DPO on NVIDIA GPU

1

I have changed some parameters in the training code as instructed, but when I run dpo on 8*A6000, I get these errors. If I understand correctly, habana is only used...

yoyo20010808

No performance difference observed using HF Tranformers Vs intel-extension-for-transformers Python API/library

3

Hello, I am running PEFT+Quantized (BitsAndBytes) Falcon model. I am trying to follow the [instructions of using Python API ](https://github.com/intel/intel-extension-for-transformers/tree/main/intel_extension_for_transformers/llm/runtime/graph#2-run-llm-with-transformer-based-api). While am able to load the model on my Intel(R)...

amir1m

## Type of Change New feature & API change ## Description - [x] Complete the pipeline to replace part of vLLM linear modules by qbits linear. (chatglm2) - [ ]...

Zhenzhong1

catch prepack error and fallback to torch bf16

1

## Type of Change bug fix ## Description a solution to discussion https://github.com/intel/intel-extension-for-transformers/discussions/1506 ## Expected Behavior & Potential Risk prepack error skipped ## How has this PR been tested? user's...

Spycsh