Kai Huang

Update: By emptying cache after each chat round, can run all the questions.

Could you provide more details? - Are you running baichuan1 or baichuan2? - What sequence lengths in and out are you using that have this memory issue?

> update new question here ， when I use interence with following gpu,how can I put inputs id to another gpu  If you have multiple GPUs, you can use...

> > > update new question here ， when I use interence with following gpu,how can I put inputs id to another gpu  > > > > > >...

> > Could you provide more details? > > > > * Are you running baichuan1 or baichuan2? > > * What sequence lengths in and out are you using...

> > > > > update new question here ， when I use interence with following gpu,how can I put inputs id to another gpu  > > > >...

Seems only one GPU is detected... Are other gpus properly set?

 You mean gpu:2 here? These two lines mean the same gpu, only one.

The changes LGTM.

Hi @K-Alex13 We have tried to use pip to install oneapi in the conda environment but failed. At this moment seems we can't put oneapi in conda env. We will...