Shengsheng Huang

> > update new question here ， when I use interence with following gpu,how can I put inputs id to another gpu  > > Then what this figure means,...

Since the problem is solved, can we close this issue?

It seems you're using OpenVINO 2023.2. For this example we only support OV 2022.3.

If you are using HuggingFace transformers to load your LLAMA model, you can refer to the llama2 example here https://github.com/intel-analytics/BigDL/tree/main/python/llm/example/CPU/HF-Transformers-AutoModels/Model/llama2 If you are using customized code to load LLAMA model,...

XLM-RoBERTa large-XNLI can be loaded using transformers API as shown in https://huggingface.co/joeddav/xlm-roberta-large-xnli#with-manual-pytorch. You can have a quick try running it using the transformers API in bigdl-llm - simply change the...

How about adding streaming llm directly into bigdl-llm instead of making it another example? It can benefit other applications /examples.

I have created a new folder for Chinese version. You can put your guide to this path: https://github.com/intel-analytics/bigdl-llm-tutorial/tree/main/Chinese_Version/ch_8_Applications.