John

Performance e-core bug(?) - only 50% CPU utilization when using all threads - (Win11, Intel 13900k)

16

I've not digged deep into this yet but my whole CPU utilization is only at 50%. I've compiled it with current VS build tools, all default, release mode of course....

Efficient preloading for mmap()

8

It's only psychological nice to have the program jump to the inference part directly after mmap load. The entire model is loaded during first inference which means it influences our...

cuBLAS - windows - static not compiling

1

When static linking is selected the CUDA::cublas_static target is not found. Dynamic binary compilation works.

struct ggml_tensor -> add a struct meta pointer into it (can replace padding)

4

Given that the tensor struct uses padding it's not nice to add any more information into it. It currently has a static 8 byte padding at the end, that's perfect...

Performance issues with cuBLAS and a bug

28

Performance with cuBLAS isn't there yet, it is more a burden than a speedup with llama eval in my tests. In a simple benchmark case it is absolutely amazing, getting...

cuBLAS memory management update

5

I changed the memory management The current variant only supports 16 allocated free buffers and it uses the first free one even if a better size is available. The new...

Feature -> tensor layer number parameter and separate from layer name

3

1) ggml tensors need a layer number parameter I'd use layer 0 for global and 1+ (could also be -1 and 0+ of course) 2) When a ggml tensor is...

### Question **Question 1)** I'm curious about the use of newline embedding, from what I can see it is appended after each of the subimage features post "unpad" then the...

Question: How can I trigger a "reload" from within my plugin ?

4

I have a problem with my plugin stopping to work for some reason, it might be LivePlugin related or something else I can't figure out. However, a click on "reload"...

Bug: AppExecutorUtil.getAppScheduledExecutorService().scheduleWithFixedDelay

4

The timer stays loaded in the context of the LivePlugin. Even when you reload (or stop) the plugin, it will still continue in the background. If you start it several...