LocalAI
LocalAI copied to clipboard
Published
4 days ago
•
mudler
mudler
Unable to find image 'localai/localai:21' locally
My first attempt to install LocalAI with install.sh and the options PORT=8081 and USE_AIO=true gave me LocalAI without images and without GPU acceleration.
I'm using Linux Mint 21, which is based on Ubuntu 22.04. I have a supported GPU:
$ nvidia-smi
Sun Jul 14 23:25:09 2024
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.256.02 Driver Version: 470.256.02 CUDA Version: 11.4 |
|-------------------------------+----------------------+----------------------+
[...]
So, I modified the install.sh this way:
[...]
. /etc/os-release
OS_NAME="ubuntu"
OS_VERSION="22.04"
info "Installing NVIDIA Container Toolkit..."
[...]
In addition, I fixed this bug:
if [ "$OS" = "Darwin" ]; then
install_binary_darwin
exit 0
fi
I replaced == with = because the use of == in the if statement within a POSIX-compliant shell (such as sh in this case) is not supported and gives me ./install.sh: 632: [: Linux: unexpected operator. We should use a single = for string comparison in a POSIX shell.
After these modifications, I ran:
sudo DOCKER_INSTALL=true PORT=8081 USE_AIO=true ./install.sh
I got:
$ sudo DOCKER_INSTALL=true PORT=8081 USE_AIO=true ./install.sh
Installing NVIDIA Container Toolkit...
Installing NVIDIA repository...
deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://nvidia.github.io/libnvidia-container/stable/deb/$(ARCH) /
#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://nvidia.github.io/libnvidia-container/experimental/deb/$(ARCH) /
Hit:1 https://ubuntu.mirror.garr.it/ubuntu jammy InRelease
Hit:2 https://packages.microsoft.com/repos/edge stable InRelease
Get:3 https://nvidia.github.io/libnvidia-container/stable/deb/amd64 InRelease [1.477 B]
Hit:4 https://dl.google.com/linux/chrome/deb stable InRelease
Get:5 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB]
Ign:6 https://linuxmint.mirror.garr.it/linuxmint/packages vanessa InRelease
Hit:7 https://ubuntu.mirror.garr.it/ubuntu jammy-updates InRelease
Hit:8 https://download.virtualbox.org/virtualbox/debian jammy InRelease
Hit:9 https://ubuntu.mirror.garr.it/ubuntu jammy-backports InRelease
Hit:10 https://linuxmint.mirror.garr.it/linuxmint/packages vanessa Release
Hit:11 https://linux.teamviewer.com/deb stable InRelease
Hit:12 https://deb.opera.com/opera-stable stable InRelease
Hit:13 https://download.sublimetext.com apt/stable/ InRelease
Get:14 https://nvidia.github.io/libnvidia-container/stable/deb/amd64 Packages [9.176 B]
Fetched 140 kB in 2s (63,7 kB/s)
Reading package lists... Done
W: https://ubuntu.mirror.garr.it/ubuntu/dists/jammy/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
W: https://ubuntu.mirror.garr.it/ubuntu/dists/jammy-updates/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
W: https://ubuntu.mirror.garr.it/ubuntu/dists/jammy-backports/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
W: https://linuxmint.mirror.garr.it/linuxmint/packages/dists/vanessa/Release.gpg: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
W: http://security.ubuntu.com/ubuntu/dists/jammy-security/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
libnvidia-container-tools libnvidia-container1 nvidia-container-toolkit-base
The following NEW packages will be installed:
libnvidia-container-tools libnvidia-container1 nvidia-container-toolkit
nvidia-container-toolkit-base
0 upgraded, 4 newly installed, 0 to remove and 0 not upgraded.
Need to get 4.471 kB of archives.
After this operation, 18,0 MB of additional disk space will be used.
Get:1 https://nvidia.github.io/libnvidia-container/stable/deb/amd64 libnvidia-container1 1.15.0-1 [924 kB]
Get:2 https://nvidia.github.io/libnvidia-container/stable/deb/amd64 libnvidia-container-tools 1.15.0-1 [19,8 kB]
Get:3 https://nvidia.github.io/libnvidia-container/stable/deb/amd64 nvidia-container-toolkit-base 1.15.0-1 [2.536 kB]
Get:4 https://nvidia.github.io/libnvidia-container/stable/deb/amd64 nvidia-container-toolkit 1.15.0-1 [992 kB]
Fetched 4.471 kB in 4s (1.079 kB/s)
Selecting previously unselected package libnvidia-container1:amd64.
(Reading database ... 742744 files and directories currently installed.)
Preparing to unpack .../libnvidia-container1_1.15.0-1_amd64.deb ...
Unpacking libnvidia-container1:amd64 (1.15.0-1) ...
Selecting previously unselected package libnvidia-container-tools.
Preparing to unpack .../libnvidia-container-tools_1.15.0-1_amd64.deb ...
Unpacking libnvidia-container-tools (1.15.0-1) ...
Selecting previously unselected package nvidia-container-toolkit-base.
Preparing to unpack .../nvidia-container-toolkit-base_1.15.0-1_amd64.deb ...
Unpacking nvidia-container-toolkit-base (1.15.0-1) ...
Selecting previously unselected package nvidia-container-toolkit.
Preparing to unpack .../nvidia-container-toolkit_1.15.0-1_amd64.deb ...
Unpacking nvidia-container-toolkit (1.15.0-1) ...
Setting up nvidia-container-toolkit-base (1.15.0-1) ...
Setting up libnvidia-container1:amd64 (1.15.0-1) ...
Setting up libnvidia-container-tools (1.15.0-1) ...
Setting up nvidia-container-toolkit (1.15.0-1) ...
Processing triggers for libc-bin (2.35-0ubuntu3.8) ...
Installing Docker...
# Executing docker install script, commit: 6d9743e9656cc56f699a64800b098d5ea5a60020
+ sh -c apt-get update -qq >/dev/null
W: http://security.ubuntu.com/ubuntu/dists/jammy-security/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
W: https://ubuntu.mirror.garr.it/ubuntu/dists/jammy/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
W: https://ubuntu.mirror.garr.it/ubuntu/dists/jammy-updates/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
W: https://linuxmint.mirror.garr.it/linuxmint/packages/dists/vanessa/Release.gpg: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
W: https://ubuntu.mirror.garr.it/ubuntu/dists/jammy-backports/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
+ sh -c DEBIAN_FRONTEND=noninteractive apt-get install -y -qq apt-transport-https ca-certificates curl >/dev/null
+ sh -c install -m 0755 -d /etc/apt/keyrings
+ sh -c curl -fsSL "https://download.docker.com/linux/debian/gpg" -o /etc/apt/keyrings/docker.asc
+ sh -c chmod a+r /etc/apt/keyrings/docker.asc
+ sh -c echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian bookworm stable" > /etc/apt/sources.list.d/docker.list
+ sh -c apt-get update -qq >/dev/null
W: http://security.ubuntu.com/ubuntu/dists/jammy-security/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
W: https://ubuntu.mirror.garr.it/ubuntu/dists/jammy/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
W: https://ubuntu.mirror.garr.it/ubuntu/dists/jammy-updates/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
W: https://linuxmint.mirror.garr.it/linuxmint/packages/dists/vanessa/Release.gpg: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
W: https://ubuntu.mirror.garr.it/ubuntu/dists/jammy-backports/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
+ sh -c DEBIAN_FRONTEND=noninteractive apt-get install -y -qq docker-ce docker-ce-cli containerd.io docker-compose-plugin docker-ce-rootless-extras docker-buildx-plugin >/dev/null
+ sh -c docker version
Client: Docker Engine - Community
Version: 27.0.3
API version: 1.46
Go version: go1.21.11
Git commit: 7d4bcd8
Built: Sat Jun 29 00:02:50 2024
OS/Arch: linux/amd64
Context: default
Server: Docker Engine - Community
Engine:
Version: 27.0.3
API version: 1.46 (minimum version 1.24)
Go version: go1.21.11
Git commit: 662f78c
Built: Sat Jun 29 00:02:50 2024
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.7.18
GitCommit: ae71819c4f5e67bb4d5ae76a6b735f29cc25774e
runc:
Version: 1.7.18
GitCommit: v1.1.13-0-g58aa920
docker-init:
Version: 0.19.0
GitCommit: de40ad0
================================================================================
To run Docker as a non-privileged user, consider setting up the
Docker daemon in rootless mode for your user:
dockerd-rootless-setuptool.sh install
Visit https://docs.docker.com/go/rootless/ to learn about rootless mode.
To run the Docker daemon as a fully privileged service, but granting non-root
users access, refer to https://docs.docker.com/go/daemon-access/
WARNING: Access to the remote API on a privileged Docker daemon is equivalent
to root access on the host. Refer to the 'Docker daemon attack surface'
documentation for details: https://docs.docker.com/go/attack-surface/
================================================================================
Starting LocalAI Docker container...
local-ai-data
Unable to find image 'localai/localai:21' locally
docker: Error response from daemon: manifest for localai/localai:21 not found: manifest unknown: manifest unknown.
See 'docker run --help'.
I'm on Linux Mint 22 and get a similar error:
$ curl https://localai.io/install.sh | sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 21509 0 21509 0 0 75931 0 --:--:-- --:--:-- --:--:-- 76003
Docker detected.
Docker detected and no installation method specified. Using Docker.
WARNING: you should run this program as super-user.
WARNING: output may be incomplete or inaccurate, you should run this program as super-user.
WARNING: you should run this program as super-user.
WARNING: output may be incomplete or inaccurate, you should run this program as super-user.
Installing NVIDIA Container Toolkit...
Could not install nvidia container toolkit - unknown OS
[sudo] password for andygeorge:
Starting LocalAI Docker container...
Unable to find image 'localai/localai:22' locally
docker: Error response from daemon: manifest for localai/localai:22 not found: manifest unknown: manifest unknown.
See 'docker run --help'.
...guessing the script assumes we're using Ubuntu, not LM.
This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
This issue was closed because it has been stalled for 5 days with no activity.