intel-extension-for-transformers

intel-extension-for-transformers copied to clipboard

Fails to load saved model : Trying to set a tensor of shape torch.Size([1376, 4096]) in "qweight" (which has shape torch.Size([4096, 1376])), this look incorrect.

8

Loading saved model runs into following error It also takes a very long time to run and save quantized models. ``` 2024-03-21 08:48:58 [INFO] loading weights file models/4_bit_llama2-rtn/model.safetensors 2024-03-21 08:48:58...

kranipa

[NeuralChat] RAG evaluation

3

1. Data augmentation: retrieval dataset construction, include (1) Context to Question and Mine Hard Negatives, (2) Context, Question to Ground Truth. 2. Retrieval evaluation: MRR (Mean reciprocal rank) and Hit...

Liangyx2

Python3.11: Could not build wheels for cchardet, which is required to install pyproject.toml-based projects

3

Ubuntu22.04 Python 3.11.9 Trying to install dependencies for NeuralChat: pip install -r requirements_cpu.txt Error: Using cached svgwrite-1.4.3-py3-none-any.whl (67 kB) Building wheels for collected packages: cchardet, lm_eval Building wheel for cchardet...

bbelky

[NeuralChat] Enable RAG's table extraction and summary

2

## Type of Change feature API changed ## Description Enable RAG's table extraction functionality for pdf Enable RAG's table summary functionality, with three modes to choose: [none, title, llm] ##...

xmx-521

[NeuralChat] Add new customized chabot UI

1

## Type of Change feature API not changed ## Description Add new customized chabot UI ## Expected Behavior & Potential Risk   ## How has this PR been tested?...

lvliang-intel

[NeuralChat] Suport pptx format for RAG

1

## Type of Change feature API not changed ## Description Support pptx format for RAG ## Expected Behavior & Potential Risk User can use pptx format file for RAG ##...

xmx-521

add FP8Config

2

## Type of Change feature ## Description do FP8 quantization using habana ## Expected Behavior & Potential Risk the expected behavior that triggered by this PR ## How has this...

mengniwang95

Support inference with WOQ and LoRA adapter

3

Hi itrex team, thanks for the great work! I've been experimenting with the Weight Only Quantization (WOQ) from ITREX, following the provided examples in [weightonlyquant.md#example-for-cpu-device](https://github.com/intel/intel-extension-for-transformers/blob/main/docs/weightonlyquant.md#example-for-cpu-device). The results are promising. Now...

Yuan0320

## Type of Change feature API added: - /v1/assist/chat - /v1/assist/decode - /v1/assist/data_transfer ## Description Support Assisted Generation on Multi-nodes. The code framework is implemented. Details will be completed by...

letonghan

system prompt can't be assigned via neuralchat frontend

5

neuralchat already synced RESTful API with latest OpenAI protocol via 2e1c79d9b99db8bc004d67235fc6df51ca1d238e But neuralchat frontend don't have field to assign system prompt. **backend log** ``` INFO: 127.0.0.1:58004 - "POST /v1/chat/completions HTTP/1.1"...

redhairerINTEL