Heyang Sun

> > Hi Yang, from our debug synch you indicated that on the same machine your fellow team member were not seeing issues on 8-GPU config. May I kindly ask...

Enabled prepare_past_kv, prepare_draft_past_kv and update_kv, and have tested on 15.5B and tiny starcoders.

Hi @jiafuzha , the error tells that an unexpected argument is passed to [BigDL-wrapped forward](https://github.com/intel-analytics/BigDL/blob/main/python/llm/src/bigdl/llm/transformers/models/mpt.py#L32) of mpt attention, and this happened because BigDL currently only supports [mosaicml/mpt-7b-chat](https://huggingface.co/mosaicml/mpt-7b-chat) and [mosaicml/mpt-30b-chat](https://huggingface.co/mosaicml/mpt-30b-chat) that...

Hi @jiafuzha , pls take a wait and I am WIP to support the feature.

Hi @jiafuzha , rotary embedding has been enabled for MPT in #10208 , you can upgrade `bigdl-llm` in your environment by `pip install --pre --upgrade bigdl-llm[all]`.

Is `import intel_extension_for_pytorch as ipex` necessary? As import will do some init works. @rnwang04

Hi @ElliottDyson , thanks for your proposal. Currently we provide many fine-tuning options e.g. ReLoRA, axolotl and DPO etc. as shown [here](https://github.com/intel-analytics/ipex-llm/tree/main/python/llm/example/GPU/LLM-Finetuning#running-llm-finetuning-using-ipex-llm-on-intel-gpu), as well as [Galore](https://github.com/intel-analytics/ipex-llm/pull/10722) and [LISA](https://github.com/intel-analytics/ipex-llm/pull/10743) on way,...

@leonardozcm pls take a look, whether it is not supported by our kernel? tks.

@Jasonzzt From the log, it is found that PPO also applies PEFT LoRA. Therefore, like QLoRA, rather than `from_pretrained` a peft model with lora config, we should first load the...

> > @caoyu-noob, you can use the `zero_to_fp32.py` script to convert the zero3 checkpoints into a regular pytorch checkpoint. You can find documentation of this script and other checkpoint conversion...