Neo Zhang Jianyu

Update INC quick start examples

1

# Existing Sample Changes ## Description 1. Update two INC quick start examples README.md with new template. 2. Fix the code error to adapt to new API. ## External Dependencies...

Update INC quick start examples

1

# Existing Sample Changes ## Description Update two INC quick start examples README.md with new template. Fix the code error to adapt to new API. Fixes Issue# ## External Dependencies...

Support get the total running time when not build caffe with CAFFE_PER_LAYER_TIMINGS := 1 in train mode. Due to the issue https://github.com/intel/caffe/issues/217 is fixed. CAFFE_PER_LAYER_TIMINGS := 1 can't be used...

RAG is slow in ChatQnA demo on Xeon

3

I setup the demo based on ChatQnA (TGI) on Xeon (GNR). Try RAG by the UI. After upload the PDF file (2-5M), I search a question. It will take 10-15s....

Optimize the LLM backend service during download LLM

2

1. When I test LLM backend service: ``` curl http://${host_ip}:9009/generate \ -X POST \ -d '{"inputs":"What is Deep Learning?","parameters":{"max_new_tokens":17, "do_sample": true}}' \ -H 'Content-Type: application/json' ``` In the first startup,...

Optimize MUL_MAT Q4_0 on Intel GPU. - Change the number of threads of kernel function. - Reorder the Q4 block to separate quantized weights and dequantize scaler. execute to reorder...

[Feature] Support Intel GPUs

6

Ollama had supported by the PR https://github.com/ollama/ollama/pull/2458 merged to support Intel GPU. But the function disappears now. I see there are several issues and opened PRs for Intel GPU. But...