Herman Semenoff

Useful fixes for good wrapper OpenCL

1

@ProjectPhysX, hi Minor optimize constructors, strict instruction to compiler default constructors and replace push to emplace.

I understand that this does not apply to project, but rather to submodule, but you do not have git submodules OpenCL configured. Therefore, I manually corrected headers for you.

ggml llama: align structs for memory optimization on 64-bit platforms

6

- ggml_type_traits_t (80 -> 72 bytes) - llama_batch (72 -> 64 bytes) - llama_model_params (56 -> 48 bytes) - hash_node (32 -> 24 bytes) - ggml_compute_state (32 -> 24 bytes)...

Does it make sense to optimize strlen in this function with for loops?

4

Function `ggml_graph_dump_dot` use 2 strlen() in ggml.c Function `ggml_visit_parents` use 2 strlen() in ggml.c Function `ggml_cpy_impl` use 1 strlen() in ggml.c Example: Before:  After: 

common, ngram_cache: added const reference for std::pair<> and std::tuple<> more 16 bytes:

2

- std::pair (72 bytes -> 8 bytes) - std::tuple (40 bytes -> 8 bytes)

grammar, json, llama: replace push on emplace if it possible

1

It rather refers to code refactoring, but there is less code for inserts functions.

ggml-opencl, llama: using reserve() if count already known

1

It affects a lot `ggml_cl_mul_mat_q_f32` function.

What is point of these statistics if most users do not set their location and stats are very inaccurate, but only approximate. I personally don't set country for blurring boundaries...