Chen Xi

hi, could you share the pretrained checkpoint or pb file , then we can check the accuracy. that would be nice.

Can you add LICENSE file to this repo?

1

I found the LICENSE file is missing in this repo, so it's hard to know the way to use this piece of work. Can you please add LICENSE?

Hi lei mao: Thanks for your blog , that benefits a lot to me as I’m enabling a style transfer model from tf-hub. I come across some problems, simply using...

add gaudi modeling support in itrex

2

## Type of Change gaudi modeling used in itrex for int4 kv-cache support

Hi gnuvince This demo is quite good for me, but i have question, how to extend the grammar to make it support define a function?

[bug] sycl_allreduce_test failed on Intel(R) Arc(TM) A770 Graphics

7

oneCCL commit: 5e7c7b7e33f5f679cb82547c4f7e49623ff0ab09 build: cmake .. -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx -DCOMPUTE_BACKEND=dpcpp run command: examples/sycl$ mpirun -n 2 ./sycl_allreduce_test gpu Log: preferred platform: Intel(R) Level-Zero, found: 8 GPU device(s) preferred platform: Intel(R) Level-Zero,...

[SYCL] fix multi-gpu issue on sycl

4

- [x] I have read the [contributing guidelines](https://github.com/ggerganov/llama.cpp/blob/master/CONTRIBUTING.md) - Self-reported review complexity: - [ ] Low - [ ] Medium - [x] High fix multi-gpu crash issue by filter the...

### System Info ```shell Optimum-habana: main dd3cc08937a85175eafd3b511484c86fed889b79 Docker images: vault.habana.ai/gaudi-docker/1.15.1/ubuntu22.04/habanalabs/pytorch-installer-2.2.0:latest DeepSpeed: main 7e475fbdcf45d763f83d47dd6c9c181be731d92e ``` ### Information - [X] The official example scripts - [ ] My own modified scripts ###...

Bug: n_ctx will reuse n_ctx_train when --ctx_size not set and make deepseek-v2 models meet out of memory crash even on a small output length.

12

### What happened? deepseek-v2 model will meet out of memory issue with the kv buffer size allocating about 43G with a 160K context length from the model. But when you...