JamieVC

streamlit iGPU -RuntimeError: Native API failed. Native API returns: -999 (Unknown PI error) -999 (Unknown PI error)

1

I followed installation guide https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Overview/install_gpu.html in order to run IPEX-LLM on iGPU of Meteor Lake with Windows OS platform. **---Steps to set environment ---** (llm) PS C:\Users\S54 PR> pip install...

[langchain-chatchat] ERROR: The expanded size of the tensor (559) must match the existing size (512) at non-singleton dimension 1. Target sizes: [1, 559]. Tensor sizes: [1, 512]

2

Hi, I download langchain-chatchat and follow README.md. After that, putting two files to ingest. ( one is PDF, and the other is txt)  It shows errors. 

Weights of LlamaForCausalLM were not initialized from the model checkpoint at meta-llama/Meta-Llama-3-8B-Instruct?

1

Here is log when running LLM with meta-llama/Meta-Llama-3-8B-Instruct. Do you know how to train this model on a down-stream task? Thanks ----- Some weights of LlamaForCausalLM were not initialized from...

default values of max_generated_tokens, top_k, top_p, and temperature?

1

What are default values of max_generated_tokens, top_k, top_p, and temperature? If user doesn't set all parameters in `generate_kwargs` such as the example below, it should use default values. How do...

Transform a string into input llama2-specific and llama3-specific input ?

1

I try to transform a string into input llama2-specific and llama3-specific input in the function `completion_to_prompt()` Is there a way to pass parameter **model_option** as a input? or else, I...

how to switch to load multiple llm models in a streamlit page?

3

I hope to switch llama2-7b-chat and llama3-8b models. But it cost a lot of memory size if I load both. How to clear one if I am going to load...

Run finetune.py on Xeon but failed - no attribute "weight"

10

### Describe the bug To finetune model on Xeon CPU, we are following the [ai-reference-models/models_v2/pytorch/llama/training/cpu at main · intel/ai-reference-models (github.com)](https://github.com/intel/ai-reference-models/tree/main/models_v2/pytorch/llama/training/cpu) · Base model: meta-llama/llama-2-7b-hf · Dataset: [raw.githubusercontent.com/tloen/alpaca-lora/main/alpaca_data.json](https://raw.githubusercontent.com/tloen/alpaca-lora/main/alpaca_data.json) ### Versions ...

Native ReAct Agent can't run on NPU channel-wise model

3

**Describe the bug** sample https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/llm-agent-react/llm-agent-react.ipynb I converted a model for NPU. (Channel-wise quantization) `optimum-cli export openvino -m Qwen/Qwen2.5-3B-Instruct --weight-format int4 --sym --ratio 1.0 --group-size -1 Qwen2.5-3B-Instruct_NPU` Then, tried to run...

notebook "llm rag llama-index" fail to initial

4

To compare multiple LLM models to OEM/ODM, we are running this sample. 1. install new env based on openvino guide: https://github.com/openvinotoolkit/openvino_notebooks/wiki/Windows 2. run >jupyter lab notebooks\llm-rag-llamaindex\llm-rag-llamaindex.ipynb 3. run the first...