jiafu zhang

@tanghaodong25 , please check.

@tanghaodong25 , I fixed the issue by referencing it from the chunk which is in-turn referenced by fi_context2. It makes sure buffer not being released after sendBuffer return. Before that,...

set use_auth_token to your actual token. it worked for me.

It's SPR node. no XPU. :~$ lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 52 bits physical, 57 bits virtual Byte Order: Little Endian CPU(s): 224 On-line CPU(s) list:...

> Hi @jiafuzha , the error tells that an unexpected argument is passed to [BigDL-wrapped forward](https://github.com/intel-analytics/BigDL/blob/main/python/llm/src/bigdl/llm/transformers/models/mpt.py#L32) of mpt attention, and this happened because BigDL currently only supports [mosaicml/mpt-7b-chat](https://huggingface.co/mosaicml/mpt-7b-chat) and [mosaicml/mpt-30b-chat](https://huggingface.co/mosaicml/mpt-30b-chat)...

As verified, rotray_emb_w_meta_info also added to mpt-7b-chat. File "/root/llm-on-ray/inference/predictor_deployment.py", line 121, in __call__ return self.predictor.generate(prompts, **config) File "/root/llm-on-ray/inference/transformer_predictor.py", line 122, in generate gen_tokens = self.model.generate( File "/opt/conda/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in...

> > Hi @jiafuzha , the error tells that an unexpected argument is passed to [BigDL-wrapped forward](https://github.com/intel-analytics/BigDL/blob/main/python/llm/src/bigdl/llm/transformers/models/mpt.py#L32) of mpt attention, and this happened because BigDL currently only supports [mosaicml/mpt-7b-chat](https://huggingface.co/mosaicml/mpt-7b-chat) and...

> Hi @jiafuzha , pls take a wait and I am WIP to support the feature. take you time. thanks.

let me try. thanks.

> @jiafuzha Do you have extra input? I was on vacation last week. Do we have any interface defined for RemoteStorageManager? I am looking forward to it since our [DAOS...