ipex-llm icon indicating copy to clipboard operation
ipex-llm copied to clipboard

Published 20 hours ago • verified publisher icon intel-analytics

Add CPU and GPU examples for DeciLM-7B

Open Zhangky11 opened this issue 1 year ago • 4 comments

Description

In this PR, we add CPU and GPU examples of DeciLM-7B using bigdl-llm's transformer and optimize_model API. This is the issue page: https://github.com/analytics-zoo/nano/issues/936

Zhangky11 avatar Jan 09 '24 07:01 Zhangky11

Please also updated the readme for Windows GPU supports as we have done here: https://github.com/intel-analytics/BigDL/pull/9992

It would be better to have some verification on Windows GPU.

Oscilloscope98 avatar Jan 29 '24 04:01 Oscilloscope98

Please also updated the readme for Windows GPU supports as we have done here: #9992

It would be better to have some verification on Windows GPU.

Sure, I'll do it.

Zhangky11 avatar Jan 29 '24 04:01 Zhangky11

I've updated README file and finished verifications on windows GPU.

Zhangky11 avatar Jan 30 '24 05:01 Zhangky11

It can work when I set cpu_embedding=True for both HF-transformer API and Pytorch API.

Zhangky11 avatar Jan 30 '24 05:01 Zhangky11