ocannl icon indicating copy to clipboard operation
ocannl copied to clipboard

Implement a Universal Pool Allocator across backends

Open lukstafi opened this issue 5 months ago • 0 comments

The Metal backend forces us to do this, but it's good for us! buffer_ptr becomes buffer_offset. With this significant refactoring, we can also decide to rename things to maybe something more consistent or standard. E.g. ctx_arrays -> ctx_buffers. Let's also use this opportunity to improve index arithmetic types. Let's support a large_models=true setting with 64 bit indexing uint64 and large_models=false with uint32 indexing.

https://claude.ai/share/2b5917b8-f103-4e28-b94a-9f0f6cfb1e88

lukstafi avatar Aug 11 '25 20:08 lukstafi