Ruonan Wang

> Are you using any metric to check for model accuracy after QLora finetuning. > I had used my custom dataset for finetuning and my inference results are not good....

Hi @xiangyang-95 , could you please share us your detailed code / error message and your pip list so that we maybe can reproduce your errors : )

By the way, based on my experience, if you want to finetune mixtral, it would be better to use transformers==4.36.1 to avoid error in saving checkpoint.

I can't reproduce this issue. Based on my test, below code will call `self.greedy_search`. ### code ```python import torch import intel_extension_for_pytorch as ipex from bigdl.llm.transformers import AutoModelForCausalLM from transformers import...

> Is `import intel_extension_for_pytorch as ipex` necessary? As import will do some init works. @rnwang04 It's not necessary, here use ipex as I validate in a GPU conda env. I...

Here is a lora finetuning script of llama for your reference : https://github.com/intel-analytics/BigDL/blob/main/python/llm/example/GPU/QLoRA-FineTuning/alpaca-qlora/lora_finetune_llama2_7b_arc_1_card.sh Based on it, if you want to finetune baichuan 13b, there needs some modifications. 1. First you...

Hi @vmadananth , pytorch xpu profiler is not supported on Windows now. You can use vtune to obtain some kernel level profile (https://www.intel.com/content/www/us/en/docs/vtune-profiler/get-started-guide/2023/windows-os.html). But now there is no accurate way...

Hi @vmadananth , I'm not sure what you mean by graph optimization here? My understanding of graph optimization seems to be for static graphs such as TF/ONNX, while IPEX-LLM [XPU]...

Hi @dan9070 & @Jrizzos , we have reproduced this error (`Error: [0] server cpu not listed in available servers map[]`) on Windows and we are trying to fix it. Once...

Hi all, this issue has been fixed for `ipex-llm[cpp]>=2.1.0b20240513` : )