[Bug] exec format error on arm64 in docker

[clarity99]

Context / Scenario

I tried to run km from docker

What happened?

docker run -e OPENAI_API_KEY="xxxx" -p 9001:9001 -it --rm kernelmemory/service got error:

exec /usr/bin/dotnet: exec format error

this is on hetzner arm cloud servers. docker inspect cd3068ce3833 of the image does contain: "Architecture": "arm64", but it seems the actual binary is not correct?

Importance

I cannot use Kernel Memory

Platform, Language, Versions

ubuntu 22.04 on arm64, image hash cd3068ce3833, hetzner cloud arm server

Relevant log output

Status: Downloaded newer image for kernelmemory/service:latest
exec /usr/bin/dotnet: exec format error

[dluc]

@cherchyk FYI

[cherchyk]

@clarity99 I'm wondering if explicitly specifying architecture would help

docker run --platform linux/arm64 -e OPENAI_API_KEY="xxxx" -p 9001:9001 -it --rm kernelmemory/service

or

docker pull --platform linux/arm64 kernelmemory/service:latest

[clarity99]

@cherchyk nope, same error. docker pull --platform linux/arm64 kernelmemory/service:latest latest: Pulling from kernelmemory/service Digest: sha256:ddcf40eea9e61aed9e857a2da9eb873ae0f5f1d8db7eda0ced57dd1bd8e816d7 Status: Image is up to date for kernelmemory/service:latest docker.io/kernelmemory/service:latest

docker run --platform linux/arm64 -e OPENAI_API_KEY="xxxx" -p 9001:9001 -it --rm kernelmemory/service exec /usr/bin/dotnet: exec format error

[dluc]

@clarity99 could you try running these 3 images, see if there's any difference?

docker pull --platform linux/arm64 kernelmemory/service:sha-3753853
docker pull --platform linux/arm64 kernelmemory/service:sha-8a2c25d
docker pull --platform linux/arm64 kernelmemory/service:sha-70d5f84

[dluc]

I can see the same error testing in Ubuntu Server 22.04 LTS, using a local VM with UTM on Apple silicon:

uname -a
Linux ubuntu 5.15.0-92-generic #102-Ubuntu SMP Wed Jan 10 09:37:39 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux

docker -v
Docker version 24.0.5, build 24.0.5-0ubuntu1~22.04.1

sudo docker pull --platform linux/arm64 kernelmemory/service:sha-3753853
sudo docker pull --platform linux/arm64 kernelmemory/service:sha-8a2c25d
sudo docker pull --platform linux/arm64 kernelmemory/service:sha-70d5f84

sudo docker images
REPOSITORY             TAG           IMAGE ID       CREATED             SIZE
kernelmemory/service   sha-3753853   a4d4e3327a9f   About an hour ago   167MB
kernelmemory/service   sha-8a2c25d   cd3068ce3833   12 days ago         167MB
kernelmemory/service   sha-70d5f84   e6bce277ceb1   2 weeks ago         163MB

sudo docker inspect kernelmemory/service:sha-70d5f84 | grep Arch
sudo docker inspect kernelmemory/service:sha-8a2c25d | grep Arch
sudo docker inspect kernelmemory/service:sha-3753853 | grep Arch
"Architecture": "arm64",
"Architecture": "arm64",
"Architecture": "arm64",

sudo docker run --platform linux/arm64 -e OPENAI_API_KEY="$OPENAI_API_KEY" -p 9001:9001 -it --rm kernelmemory/service:sha-70d5f84
exec /usr/bin/dotnet: exec format error

sudo docker run --platform linux/arm64 -e OPENAI_API_KEY="$OPENAI_API_KEY" -p 9001:9001 -it --rm kernelmemory/service:sha-8a2c25d
exec /usr/bin/dotnet: exec format error

sudo docker run --platform linux/arm64 -e OPENAI_API_KEY="$OPENAI_API_KEY" -p 9001:9001 -it --rm kernelmemory/service:sha-3753853
exec /usr/bin/dotnet: exec format error

Testing AMD64, just in case:

sudo docker run --platform linux/amd64 -e OPENAI_API_KEY="$OPENAI_API_KEY" -p 9001:9001 -it --rm kernelmemory/service:sha-70d5f84
exec /usr/bin/dotnet: exec format error

[cherchyk]

Give us additional time to find solution

uname -a
Linux Laptop-3 5.15.133.1-microsoft-standard-WSL2 #1 SMP Thu Oct 5 21:02:42 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

sudo docker run --platform linux/amd64 -e OPENAI_API_KEY="sadf" -p 9001:9001 -it --rm kernelmemory/service:sha-70d5f84
***************************************************************************************************************************
* Environment         : Production
* Web service         : Enabled
* Web service auth    : Disabled
* Pipeline handlers   : Enabled
* OpenAPI swagger     : Disabled
* Logging level       : Information
* Memory Db           : Microsoft.KernelMemory.MemoryStorage.DevTools.SimpleVectorDb
* Content storage     : Microsoft.KernelMemory.ContentStorage.DevTools.SimpleFileStorage
* Embedding generation: Microsoft.KernelMemory.AI.OpenAI.OpenAITextEmbeddingGenerator
* Text generation     : Microsoft.KernelMemory.AI.OpenAI.OpenAITextGenerator
***************************************************************************************************************************
info: Microsoft.KernelMemory.ServiceAssembly[0]
      Starting Kernel Memory service, .NET Env: Production, Log Level: Information, Web service: True, Auth: False, Pipeline handlers: True

[dluc]

As of today, the error is still occurring.

Linux ubuntu 5.15.0-92-generic #102-Ubuntu SMP Wed Jan 10 09:37:39 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux

sudo docker run --platform linux/amd64 -e OPENAI_API_KEY="xxx" -p 9001:9001 -it --rm kernelmemory/service exec /usr/bin/dotnet: exec format error

sudo docker run --platform linux/arm64 -e OPENAI_API_KEY="xxx" -p 9001:9001 -it --rm kernelmemory/service exec /usr/bin/dotnet: exec format error

[dluc]

I recently reviewed the Dockerfile with support from the Docker team, and looks like there's some error in the use of build platform and target platform. The error can be reproduced running a Linux VM inside an ARM-based macbook.

[dluc]

We've refactored the images, to workaround some limitations of the current CI pipeline.

The latest label is now used only for Intel/AMD CPUs. To run KM docker image on ARM64, e.g. Linux ARM, new Mac laptops, please use the latest-arm64 label.

So, about the initial post, the command would be:

docker run -e OPENAI_API_KEY="xxxx" -p 9001:9001 -it --rm kernelmemory/service:latest-arm64

[dluc]

Update: the latest tag now contains both AMD64 and ARM64 images, using docker arch autodetect. As a result the latest-arm64 has been deprecated.