bigdl-llm-tutorial

bigdl-llm-tutorial copied to clipboard

GPU acceleration failed

1

I use the code here： https://github.com/intel-analytics/ipex-llm-tutorial/blob/original-bigdl-llm/Chinese_Version/ch_6_GPU_Acceleration/6_1_GPU_Llama2-7B.md But failed. Can you help with this? Thanks. `from bigdl.llm.transformers import AutoModelForCausalLM, AutoModel from transformers import LlamaTokenizer, AutoTokenizer chatglm3_6b = 'D:/AI_projects/Langchain-Chatchat/llm_model/THUDM/chatglm2-6b' model_in_4bit = AutoModel.from_pretrained(pretrained_model_name_or_path=chatglm3_6b,...

doubtfire009

XPU inference failed

1

My code is based on bigdl-llm `from langchain import LLMChain, PromptTemplate from bigdl.llm.langchain.llms import TransformersLLM from langchain.memory import ConversationBufferWindowMemory chatglm3_6b = 'D:/AI_projects/Langchain-Chatchat/llm_model/THUDM/chatglm3-6b' llm_model_path = chatglm3_6b # huggingface llm 模型的路径 CHATGLM_V3_PROMPT_TEMPLATE...

doubtfire009

❯ pip install --pre --upgrade ipex-llm[all] zsh: no matches found: ipex-llm[all]

meijiesky

- The `output_path` in sample python code is missing quotes - `torch` isn't properly imported - `tokenizer` should be loaded before loading dataset

Mingyu-Wei

Replace the deprecated demo audio files/voice dataset

Mingyu-Wei

Python performance is too poor, can it provide an inference library in C++ and provide an OpenAI-compatible API

3

Using bigdl-llm in a production environment, Python performance is too poor, can you provide an inference library in C++ and provide an OpenAI-compatible API

geffzhang

GPU installation need to be updated

1

We need to update GPU installation (including PyTorch2.1 support, Windows installation) in chapter6 and 7, referring to https://bigdl.readthedocs.io/en/latest/doc/LLM/Overview/install_gpu.html

plusbang

about the memory problem

3

 Each time when I interact with the model, the memory occupied by the model increases and does not release memory resources. As a result, when there are many conversations,...

K-Alex13

About the acclerate problem with xpu

2

 After putting the model and inputs to xpu, the model is work now on intel laptop. But the inference time is about 588 seconds that is too long for...

K-Alex13

Update Chapter 5 5_1_ChatBot and 5_1_2_Speech Recognition notebook in the English version and Chinese version

NovTi