ollama service

[cloudlena]

It would be amazing to have a services.ollama module in home manager, just like we have it in NixOS.

[terlar]

I copied it over a while back and it seems to work quite well, here is the code in case you want a base:

{
  config,
  lib,
  pkgs,
  ...
}: let
  inherit (lib) types;

  cfg = config.services.ollama;
  ollamaPackage = cfg.package.override {
    inherit (cfg) acceleration;
  };
in {
  options = {
    services.ollama = {
      enable = lib.mkEnableOption (
        lib.mdDoc "Server for local large language models"
      );
      listenAddress = lib.mkOption {
        type = types.str;
        default = "127.0.0.1:11434";
        description = lib.mdDoc ''
          Specifies the bind address on which the ollama server HTTP interface listens.
        '';
      };
      acceleration = lib.mkOption {
        type = types.nullOr (types.enum ["rocm" "cuda"]);
        default = null;
        example = "rocm";
        description = lib.mdDoc ''
          Specifies the interface to use for hardware acceleration.

          - `rocm`: supported by modern AMD GPUs
          - `cuda`: supported by modern NVIDIA GPUs
        '';
      };
      package = lib.mkPackageOption pkgs "ollama" {};
    };
  };

  config = lib.mkIf cfg.enable {
    systemd.user.services.ollama = {
      Unit = {
        Description = "Server for local large language models";
        After = ["network.target"];
      };

      Service = {
        ExecStart = "${lib.getExe ollamaPackage} serve";
        Environment = ["OLLAMA_HOST=${cfg.listenAddress}"];
      };

      Install = {WantedBy = ["default.target"];};
    };

    home.packages = [ollamaPackage];
  };
}

[rycee]

Yep, feel free to open a PR with that. Please remove the unnecessary lib.mdDoc calls, and separate the option definitions by empty lines.

[jcrqr]

@terlar sorry for the noob question, how can I add this to an existing flake?

I've created a new file with the configuration you provided (thanks!), should I add it in my home-manager configuration or import it in other special place?

I don't have my configuration published at the moment but will do so tomorrow if you want to check it.

Thanks in advance.

[terlar]

@wtfiscrq Yes, it is part of the Home Manager configuration. What I posted is the interface and after that you have to configure/use it in order to use it.

So either refer to the file directly inside the homeManagerConfiguration function as such:

...
      inputs.home-manager.lib.homeManagerConfiguration {
        inherit pkgs;
        modules = [
          ./path/to/ollama-service-definition.nix
          {
            services.ollama = {
              enable = true;
              acceleration = "cuda";
            };
          }
        ];
      }
...

or if you have another file of Home Manager configuration you can do:

{
  imports = [ ./path/to/ollama-service-definition.nix ];
  services.ollama = {
    enable = true;
    acceleration = "cuda";
  };
}

[stale[bot]]

Thank you for your contribution! I marked this issue as stale due to inactivity. Please be considerate of people watching this issue and receiving notifications before commenting 'I have this issue too'. We welcome additional information that will help resolve this issue. Please read the relevant sections below before commenting.

If you are the original author of the issue

• If this is resolved, please consider closing it so that the maintainers know not to focus on this.
• If this might still be an issue, but you are not interested in promoting its resolution, please consider closing it while encouraging others to take over and reopen an issue if they care enough.
• If you know how to solve the issue, please consider submitting a Pull Request that addresses this issue.

If you are not the original author of the issue

• If you are also experiencing this issue, please add details of your situation to help with the debugging process.
• If you know how to solve the issue, please consider submitting a Pull Request that addresses this issue.

Memorandum on closing issues

Don't be afraid to manually close an issue, even if it holds valuable information. Closed issues stay in the system for people to search, read, cross-reference, or even reopen – nothing is lost! Closing obsolete issues is an important way to help maintainers focus their time and effort.

[terlar]

I spent some extra time trying to get this in a state ready for contribution. PR now opened.