Eric Curtin

Pull models from OCI registries with a specific UserAgent

6

A reference implementation: https://github.com/ggml-org/llama.cpp/pull/15790/files Please, implement with a LocalAI specific User-Agent string so we can see stats on who is using what client-side tool. The llama-server User-Agent string is "llama.cpp".

Refactor: Replace manual memory management with smart pointers

2

Replaced all `malloc`/`free` calls with `std::unique_ptr` to leverage RAII for memory management. Used custom deleters where needed to handle specific free functions, such as `stbi_image_free` and `free_sd_ctx`. Simplified resource cleanup...

Enhanced docker support for "bootc switch"

3

On my personal machine, been trying to use docker with bootc, we need to enhance to use docker's container storage somehow: sudo bootc switch --transport containers-storage "$id" Any opinions on...

A reference implementation: https://github.com/ggml-org/llama.cpp/pull/15790/files Please, implement with a lmstudio specific User-Agent string so we can see stats on who is using what client-side tool. The llama-server User-Agent string is "llama.cpp".

AI models in microVMs

17

### Feature request description We either already have some of this functionality: https://github.com/microsandbox/microsandbox It should be just: ``` ramalama run --oci-runtime krun smollm:135m ``` Needs testing to ensure GPU access...

Could you post a video demo somewhere?

1

I'd love to see a video of this all used together here on youtube or something FWIW 😄 Docker Model Runner + docker-compose + docling + ai/granite-4.0-h-tiny

Investigate KosmicKrisp

2

Maybe it's better than MoltenVK, who knows? https://www.lunarg.com/a-vulkan-on-metal-mesa-3d-graphics-driver/