Borislav Stanimirov
Borislav Stanimirov
Well, I have talk on the subject. You can check it out: https://www.youtube.com/watch?v=bbbcZd4cuxg
> I think this can be handled by using functions to create custom buffers, such as `ggml_backend_cpu_buffer_from_ptr`, and adding a function to allocate a tensor at a specific offset from...
Why not create a dummy buffer with size zero, which can be used for all external tensors for a given backend? Thus the backend will be accessible, but allocating data...
@slaren, how's this?
> We cannot just add a dummy buffer to the backends, this may not work in every backend. Why not? It's a dummy buffer anyway. If for some reason a...
> `ggml_backend_cuda_set_tensor_external_data` is unnecessary and duplicates the code from `init_tensor`. The key part here is that the extra is reused if available and `ggml_backend_cuda_buffer_init_tensor` sets a new one every time....
I think there is some misunderstanding here. The dummy buffer is just a dummy. It should never be used to actually allocate. It's there just to signify that the tensor...
Wouldn't potential extra data like buffer objects, be stored in that backend's `extra` member per tensor? I still don't quite get what you mean. Say I have an OpenGL backend:...
Though it does make sense to have `offset` as a future proof arg for `set_external_data` for cases where data is totally opaque. I'm adding it now
I found a problem with the dummy buffer for cuda. Since it is impossible to associate something with the lifetime of a tensor, it is impossible to manage the extras...