ocannl
ocannl copied to clipboard
Implement a Universal Pool Allocator across backends
The Metal backend forces us to do this, but it's good for us! buffer_ptr becomes buffer_offset.
With this significant refactoring, we can also decide to rename things to maybe something more consistent or standard.
E.g. ctx_arrays -> ctx_buffers.
Let's also use this opportunity to improve index arithmetic types. Let's support a large_models=true setting with 64 bit indexing uint64 and large_models=false with uint32 indexing.
https://claude.ai/share/2b5917b8-f103-4e28-b94a-9f0f6cfb1e88