Server tls 3203

[gabe-l-hart]

Disclaimer!

This PR started as a small feature addition and resulted in some significant scope creep when I added the unit tests. I'm certainly open to trying to remove some of that refactoring for ServerNonBlocking if that's preferred, but figured it was worth presenting as-is to start the discussion.

Issues

This issue supports https://github.com/ollama/ollama/issues/3203 by adding encryption between client and server. It does not fully address the issue since the core feature request is for auth.

Description

This PR adds support for running the primary ollama server/client interactions using TLS and Mutual TLS (mTLS). It does not address encryption between the ollama server and the individual model servers.

Changes

The changes in this PR are grouped as follows:

New envconfig variables:

• To boot an (m)TLS server:
  • OLLAMA_HOST: If the scheme is https://, the server will attempt to boot with TLS or mTLS based on the presence of the below variables
  • OLLAMA_TLS_SERVER_KEY: File with the private key (required for TLS)
  • OLLAMA_TLS_SERVER_CERT: File with the public cert (required for TLS)
  • OLLAMA_TLS_CLIENT_CA: File with the CA cert for the key/cert pair the client will use (required for mTLS)
• To connect a client to an (m)TLS server:
  • OLLAMA_HOST: If the scheme is https://, the client will attempt to connect to a (m)TLS server depending on the presence of the below variables
  • OLLAMA_TLS_SERVER_CA: File with the CA cert for the server's key/cert pair (required for TLS if the server's key/cert pair is signed by a non-system CA. If not given, but the scheme is https://, the system CAs will be used.)
  • OLLAMA_TLS_CLIENT_KEY: File with the private key for the client when connecting to an mTLS server (required for mTLS)
  • OLLAMA_TLS_CLIENT_CERT: File with the public cert for the client when connecting to an mTLS server (required for mTLS)

Config parsing:

• In envconfig, there is a new getTlsConfig function which parses all of the TLS-related variables for both client and server
• getTlsConfig is used to populate envconfig.ServerTlsConfig and envconfig.ClientTlsconfig with tls.Config objects if configured to use (m)TLS
• If not configured for TLS, these objects will remain nil which is the indicator elsewhere in the code that TLS is not enabled

Server Setup

The primary change in routes.go is to add a conditional around calling the ServeTLS function on the http.Server object based on the value of envconfig.ServerTlsConfig.

The rest of the changes there were all made in support of helping to make the server easier to boot in unit tests. For that, I split the Serve function into two parts: ServeNonBlocking which returns an instance of the server.Server struct, and Serve which uses ServeNonBlocking and then blocks on the server terminating.

Client Setup

In client.go, the change is to look at envconfig.ClientTlsConfig and instantiate the http.Client accordingly

Unit Testing

• Add a new testing package envconfig/configtest that holds helpers for dynamically generating TLS data
• Extend the envconfig tests to test the parsing of config data
• Add a new server/mtls_test.go test suite that tests the server/client communication with no TLS, standard TLS, and mTLS

Testing

In addition to the unit tests, I've also verified that the communication works separately using scripts I have from other projects for generating self-signed mTLS data. Here are the steps I used:

gen_mtls_test_files.sh

#!/usr/bin/env bash

## Config ######################################################################

# Optional additional SANs can be set with SANS
SANS=${SANS:-""}

# CN can be overloaded
CN=${CN:-"foo.bar.com"}

set -eo pipefail

# Set up additional SANs block
IFS=' ' read -r -a sans_arr <<< "$SANS"
extra_sans=""
counter="1"
for san in "${sans_arr[@]}"
do
    counter=$(expr "$counter" "+" "1")
    echo "Adding SAN DNS.$counter [$san]"
    extra_sans="$extra_sans\nDNS.$counter = $san"
done

root_name="ca"
root_key="$root_name.key.pem"
root_crt="$root_name.cert.pem"

server_name="server"
server_key="$server_name.key.pem"
server_crt="$server_name.cert.pem"

client_name="client"
client_key="$client_name.key.pem"
client_crt="$client_name.cert.pem"

common_config='
[req]
default_bits       = 4096
default_keyfile    = server.key.pem
distinguished_name = req_distinguished_name
x509_extensions    = x509_ext
string_mask        = utf8only

[req_distinguished_name]
countryName                 = Country Name (2 letter code)
countryName_default         = US
stateOrProvinceName         = State or Province Name (full name)
stateOrProvinceName_default = Denver
localityName                = Locality Name (eg, city)
localityName_default        = Denver
organizationName            = Organization Name (eg, company)
organizationName_default    = Gabe Inc
commonName                  = Common Name (eg, YOUR name)
commonName_max              = 64
'

ca_config="
$common_config

[x509_ext]
subjectKeyIdentifier   = hash
authorityKeyIdentifier = keyid:always, issuer
basicConstraints       = critical, CA:TRUE, pathlen:1
keyUsage               = keyCertSign, cRLSign
"

derived_config="
$common_config

[x509_ext]
subjectKeyIdentifier   = hash
authorityKeyIdentifier = keyid,issuer
basicConstraints       = CA:FALSE
keyUsage               = digitalSignature, keyEncipherment
subjectAltName         = @alt_names

[alt_names]
DNS.1 = localhost
$extra_sans
IP.1  = '127.0.0.1'
"

# use wild card in subject, not all clients accept that, but Java grpc client does
# we also have subject alternative names 127.0.0.1 and localhost in our openssl.cnf file (used when creating the server crt)
SUBJ="/C=US/ST=Denver/L=Denver/O=Gabe Inc/CN=$CN"

# Set the expiration for 10 years
expiration_days=3650

## Root ########################################################################


# Create the root key
echo "[Creating root key]"
openssl genrsa -out $root_key 2048

# create root key and cert
echo "[Creating root cert]"
openssl req \
    -config <(echo -e "$ca_config") \
    -x509 \
    -new \
    -nodes \
    -key $root_key \
    -sha256 \
    -subj "$SUBJ" \
    -out $root_crt

## Server ######################################################################

# create a new server key and certificate signing request
echo "[Creating server key and signing request]"
openssl req -config <(echo -e "$derived_config") -new -nodes -sha256 -keyout $server_key -out $server_name.csr -newkey rsa:2048 -subj "$SUBJ"

# sign the request with our root cert key
echo "[Sign server cert]"
openssl x509 -req -sha256 -extfile <(echo -e "$derived_config") -extensions x509_ext -in $server_name.csr -CA $root_crt -CAkey $root_key -CAcreateserial -out $server_crt -days $expiration_days

# write out server key in pkcs8 format, required by grpc
echo "[Convert server key to pkcs8]"
cp $server_key $server_key.tmp
openssl pkcs8 -topk8 -nocrypt -in $server_key.tmp -out $server_key

## Client ######################################################################

# create a new server key and certificate signing request
echo "[Creating client key and signing request]"
openssl req -config <(echo -e "$derived_config") -new -nodes -sha256 -keyout $client_key -out $client_name.csr -newkey rsa:2048 -subj "$SUBJ"

# sign the request with our root cert key
echo "[Sign client cert]"
openssl x509 -req -sha256 -extfile <(echo -e "$derived_config") -extensions x509_ext -in $client_name.csr -CA $root_crt -CAkey $root_key -CAcreateserial -out $client_crt -days $expiration_days

# write out client key in pkcs8 format, required by grpc
echo "[Convert client key to pkcs8]"
cp $client_key $client_key.tmp
openssl pkcs8 -topk8 -nocrypt -in $client_key.tmp -out $client_key

# Clean up
rm *.tmp
rm *.csr
rm *.srl

# Using the above gen_mtls_test_files.sh
gen_mtls_test_files.sh

# Build it
go build .

# Boot the server with mTLS enabled
OLLAMA_HOST="https://localhost:54321" \
OLLAMA_TLS_SERVER_KEY=server.key.pem \
OLLAMA_TLS_SERVER_CERT=server.cert.pem \
OLLAMA_TLS_CLIENT_CA=ca.cert.pem \
./ollama serve

# Run a client command with mTLS enabled (separate terminal or background the server)
OLLAMA_HOST="https://127.0.0.1:54321" \
OLLAMA_TLS_SERVER_CA=ca.cert.pem \
OLLAMA_TLS_CLIENT_KEY=client.key.pem \
OLLAMA_TLS_CLIENT_CERT=client.cert.pem  \
./ollama ls