intel-extension-for-transformers

intel-extension-for-transformers copied to clipboard

SageMaker does not support Transformers 4.34.1 which is required by ITREX

3

ValueError: Unsupported huggingface version: 4.34.1. You may need to upgrade your SDK version (pip install -U sagemaker) for newer huggingface versions. Supported huggingface version(s): 4.6.1, 4.10.2, 4.11.0, 4.12.3, 4.17.0, 4.26.0,...

eduand-alvarez

AssertionError: Fail to convert pytorch model with 'Intel/neural-chat-7b-v3-3' WOQ

2

```python from transformers import AutoTokenizer, TextStreamer from intel_extension_for_transformers.transformers import AutoModelForCausalLM, WeightOnlyQuantConfig model_name = "Intel/neural-chat-7b-v3-3" # for int8, should set weight_dtype="int8" config = WeightOnlyQuantConfig(compute_dtype="bf16", weight_dtype="int4") prompt = "Once upon a time,...

eduand-alvarez

Llava Models

7

Does the Llava part work ? https://github.com/intel/intel-extension-for-transformers/tree/main/intel_extension_for_transformers/transformers/modeling/llava_models If so are they optimized for Intel Device and are there any examples ? Thanks for building this library. I have seen the...

choochtech

feature request: support for TextIterationstreamer of HF

4

Hi I want to be able to stream the model not only to stdoutput. the current streamer: TextStreamer only works with the stdouput as I understands it. I tried using...

RachelShalom

422 Unprocessable Entity using Neural Chat via OpenAI interface with meta--lama/llama-2-7b-chat-hf

2

Is there a specific version of openai that is aligned with the OpenAI interfaces offered by neuralchat? I am currently testing using the current **1.12.0** but encountering a **422 Unprocessable...

brent-elliott

Device does not exist / is not supported error with neuralchat deploy_chatbot_on_xpu notebook

6

### Problem Summary and status of similar tests I am having trouble getting neuralchat to work with my **Intel Data Center Flex 170 GPU**. Below is my procedure with the...

brent-elliott

[NeuralChat] Support user management in backend server

2

## Type of Change feature API not changed ## Description Support user management in backend server ## Expected Behavior & Potential Risk Support user management in backend server ## How...

lvliang-intel

main example for qLoRA fails: AttributeError: 'Model' object has no attribute 'named_parameters'

4

Running the short example found in the main doc: https://github.com/intel/intel-extension-for-transformers/blob/main/docs/qloracpu.md But it crashes here: ``` model = prepare_model_for_kbit_training(model, use_gradient_checkpointing = True) File "/home/xtof/nvme/envs/cpuqlora/lib/python3.9/site-packages/peft/utils/other.py", line 95, in prepare_model_for_kbit_training for name, param...

cerisara

Installed intel-extension-for-transformers and I get an error - No module named 'intel_extension_for_pytorch'"

6

I am using Arc770 GPU on Windows 11 1. I have installed WSL2 2. I have installed miniconda 3. I follow instruction - "pip install intel-extension-for-transformers" 4. Run the example...

sungkim11

## Type of Change Add NeuralChat example API not changed ## Description Add Multi-Socket LLM inference example for NeuralChat. Related DeepSpeed PR: https://github.com/microsoft/DeepSpeed/pull/4750 (not merged yet) ## Expected Behavior &...

letonghan