ollama

ollama copied to clipboard

Adds strictly confined snap packaging for x86-64 (~~and arm64~~ just x86-64 for starters, looks like this needs overall a bit of love in ollama), presently published on the channel latest/beta. This is a nice alternative to docker (no need to install and configure the nvidia docker runtime for example, systemd service is set up automatically, over-the-air updates, straightforward to access resources and data from user's host system within the limits of the application's confinement) and safer than bare installation onto host system with the shell script that some users might not want to go ahead with (strict confinement ~= containerised analogously to docker from the host system).

Installable with:

sudo snap install ollama --channel latest/beta

• strict confinement used with network, network-bind, home, removable-media, opengl interfaces in use, i.e. it can access and serve a port, access home directory and /media, and access the GPU (the opengl interface also grants access to CUDA etc).
• starts up a systemd service automatically with ollama serve.
• if removable media access is needed (e.g. user prefers storing models under a disk mounted under /media), sudo snap connect ollama:removable-media (for security reasons, removable media access not granted without user action).

If this looks interesting, I'm happy to hand over the package on snapcraft.io to an ollama maintainer, and can contribute CI integration to make it easy to keep the snap package up to date whenever you release.

If you want to build this locally, after installing snapcraft and either the multipass or LXD provider for it go to the root directory of the repository, and ...:

snapcraft

Configuration

• host configurable in style sudo snap set ollama host=0.0.0.0:12345 (changing the config value will automatically restart the systemd service)
• models directory configurable in style sudo snap set ollama models=/your/preferred/path/to/your/models (changing the config value will automatically restart the service)
• when calling ollama from the shell, automatically calls it with OLLAMA_HOST and OLLAMA_MODELS set based on above configuration (i.e. no need for setting these in bashrc etc).

By commit f576d3e CUDA support tested and works alright. Need to test next with rocm...