ddpasa

> I got better results after resizing or cropping reference image and densepose video to 512x512. Indeed. The readme does not make it very clear, but you should be using...

Thanks, I'm seeing the same! I thought there was something wrong in my ollama implementation, but looks like there isn't and this is a problem with the vulkan backend in...

I think the bug is confirmed by now. Tagging @ggerganov , maybe he knows the origin of this behaviour.

> Make sure all ops used in `llava` are exercised in `test-backend-ops` and the tests with Vulkan backend are passing Thanks! I'll take a look. Can you give me the...

I tried to run CLIP with a Vulkan backend, and it failed with: `ggml_vulkan: Error: Missing op: ACC` It looks like the Vulkan backend does not support this operation needed...

Continuing to debug this. Enabling GGML_VULKAN_CHECK_RESULTS causes compilation to fail with: ``` llama.cpp/ggml-vulkan.cpp: In function ‘void ggml_vk_check_results_1(ggml_backend_vk_context*, ggml_compute_params*, ggml_tensor*)’: llama.cpp/ggml-vulkan.cpp:5744:61: error: base operand of ‘->’ has non-pointer type ‘vk_buffer_ref’ {aka...

> Continuing to debug this. Enabling GGML_VULKAN_CHECK_RESULTS causes compilation to fail with: > > ``` > llama.cpp/ggml-vulkan.cpp: In function ‘void ggml_vk_check_results_1(ggml_backend_vk_context*, ggml_compute_params*, ggml_tensor*)’: > llama.cpp/ggml-vulkan.cpp:5744:61: error: base operand of ‘->’...

> @ddpasa Thank you for looking into this and apologies for not replying to this issue earlier. > > I haven't personally tested llava at all, I think there are...

@0cc4m is there anything else we can do to help debug this? It looks like tha MUL_MATs have some error in them, but not sure if that is the root...

@0cc4m , I just tried again with the latest mainline (1b67731) and the behaviour is much better. What was the issue? I'll run a more detailed test in the following...