shell-pilot icon indicating copy to clipboard operation
shell-pilot copied to clipboard
Published 10 months ago • verified publisher icon reid41

Running S-Pilot as Containers

Open varkrish opened this issue 1 year ago • 3 comments

Added:

  1. Containerised version of ollama (with or without models)
  2. Containarisation scripts for s-pilot
  3. Compose for running the whole stack :)

varkrish avatar Apr 15 '24 07:04 varkrish

@varkrish thank you so much for your contribution. So far, I found some issues, it might need you help to fix it:

  1. Seem only ollama directory, not see ollama-server
  2. Not sure whether my version is different, I used rhel 8 install podman and podman-compose, seems not see built, but build and podman-compose, not see podman compose
# rpm -qa|grep podman
podman-gvproxy-4.6.1-8.module+el8.9.0+21525+acb5d821.x86_64
podman-plugins-4.6.1-8.module+el8.9.0+21525+acb5d821.x86_64
podman-4.6.1-8.module+el8.9.0+21525+acb5d821.x86_64
podman-catatonit-4.6.1-8.module+el8.9.0+21525+acb5d821.x86_64
podman-compose-1.0.6-1.el8.noarch
  1. Tried several times with MODEL=ollama, it always failed with below, but not for llama2, seems something wrong with the logic
# podman built -t ollama-server .  --build-arg PULL_MODEL_BY_DEFAULT=true --build-arg MODEL=ollama

pulling manifest ⠇ [GIN] 2024/04/15 - 13:51:18 | 200 |  967.146405ms |       127.0.0.1 | POST     "/api/pull"
pulling manifest
Error: pull model manifest: file does not exist
Error: building at STEP "RUN if [ "$PULL_MODEL_BY_DEFAULT" = "true" ]; then ollama serve & sleep 50 && ollama pull $MODEL; fi": while running runtime: exit status 1

# podman build -t ollama-server .  --build-arg PULL_MODEL_BY_DEFAULT=true --build-arg MODEL=llama2
pulling manifest
pulling 8934d96d3f08...  18% ▕██              ▏ 686 MB/3.8 GB   71 MB/s     44stime=2024-04-15T13:52:43.371Z level=INFO source=download.go:250 msg="8934d96d3f08 part 36 stalled; retrying"
time=2024-04-15T13:52:43.371Z level=INFO source=download.go:250 msg="8934d96d3f08 part 26 stalled;
  1. May I know what is the purpose of the no model steps? Pull it by user?
To build the ollama-server without any model
cd ollama-server
podman built -t ollama-server . 
Note: Model will be downloaded at during the container is started if the model is not available

Step 1: Build the ollama-server with embedded models
  1. Seems will run by podman-compose, can we put it in the one directory, it mean when user want to user container model.
compose.yml
Containerfile
  1. May be is a little challenge， but seems easy for you. Actually, is it possible to make it simple?? E.g, one command with podman-compose, and then done, easy to use
  2. It would be better provide some small steps to troubleshoot after deploy, like enter container, install vi rpm to view, how to check in server
<<You>>
hi

<<ShellPilot>>
  1. It would be better you can make more test in some Linux hosts.
  2. It would be better the container can follow the installation format
### Running the service as a container
#### Ollama Server and S-Pilot Podman Compose Configuration

or ### Running the service as a container: Ollama Server and S-Pilot Podman Compose Configuration
  1. Actually, one of the interesting thing is s-pilot can interact with the current system and help to manage the system, not just the chat mode, so in the container, seems only can use chat mode.

reid41 avatar Apr 15 '24 14:04 reid41

@reid41

  1. ollama-server is the service name created out of the scripts inside ollama folder.
  2. built is now build. Fixed typo.
  3. compose.yml now has the build instructions so we don't need to build individually. But image built is required if we make code changes.
  4. Container can interact with local volumes through volumes definition in the compose.

varkrish avatar Apr 16 '24 02:04 varkrish

hi @varkrish

  1. podman compose is podman-compose???
# podman compose up -d
Error: unknown shorthand flag: 'd' in -d
See 'podman --help'
  1. directory: is ollama or ollama-server
cd ollama-server <<<
[root@rhel8u7 ~]# ls -l shell-pilot/
total 64
-rw-r--r--. 1 root root   370 Apr 16 21:58 Containerfile
-rw-r--r--. 1 root root  1062 Apr 16 21:58 LICENSE
-rw-r--r--. 1 root root 13702 Apr 16 21:58 README.md
-rw-r--r--. 1 root root  1422 Apr 16 21:58 compose.yml
drwxr-xr-x. 2 root root     6 Apr 16 21:59 models
drwxr-xr-x. 2 root root    72 Apr 16 21:58 ollama  <<<-----
-rw-r--r--. 1 root root 22645 Apr 16 21:58 s-pilot
-rw-r--r--. 1 root root  1793 Apr 16 21:58 spilot_common.sh
-rw-r--r--. 1 root root  5919 Apr 16 21:58 spilot_install.sh
  1. I mean for container mode, is it able to put it all files in one directly??? ollama or ollama-server directly?
[root@rhel8u7 ~]# ls -l shell-pilot/Containerfile  shell-pilot/compose.yml
-rw-r--r--. 1 root root  370 Apr 16 21:58 shell-pilot/Containerfile
-rw-r--r--. 1 root root 1422 Apr 16 21:58 shell-pilot/compose.yml

# ls -l shell-pilot/
total 64
-rw-r--r--. 1 root root   370 Apr 16 21:58 Containerfile
-rw-r--r--. 1 root root  1062 Apr 16 21:58 LICENSE
-rw-r--r--. 1 root root 13702 Apr 16 21:58 README.md
-rw-r--r--. 1 root root  1422 Apr 16 21:58 compose.yml
drwxr-xr-x. 2 root root     6 Apr 16 21:59 models
drwxr-xr-x. 2 root root    72 Apr 16 21:58 ollama
-rw-r--r--. 1 root root 22645 Apr 16 21:58 s-pilot
-rw-r--r--. 1 root root  1793 Apr 16 21:58 spilot_common.sh
-rw-r--r--. 1 root root  5919 Apr 16 21:58 spilot_install.sh
  1. Could you please try to make it easier and detail? I just read the steps, I still don't understand which step is necessary, which is not need. after deployed, how to run the command in container, maybe I don't have more knowledge about container.

But I mean we can make it easier for the user. Hope u can understand.

[root@rhel8u7 ollama]# podman ps -a
CONTAINER ID  IMAGE                           COMMAND         CREATED         STATUS                     PORTS                     NAMES
2ef9f37a895d  localhost/ollama-server:latest                  23 minutes ago  Exited (1) 23 minutes ago  0.0.0.0:11434->11434/tcp  shell-pilot_ollama-server_1
41f0dd29e963  localhost/spilot:latest         sleep infinity  23 minutes ago  Up 23 minutes (healthy)                              shell-pilot_s-pilot_1

root@rhel8u7 ollama]# podman exec -it 41f0dd29e963 /bin/bash
bash-4.4$ ll
bash: ll: command not found
bash-4.4$ ls -l
total 64
-rwxrwxr-x. 1 root root   370 Apr 16 13:58 Containerfile
-rwxrwxr-x. 1 root root  1062 Apr 16 13:58 LICENSE
-rwxrwxr-x. 1 root root 13702 Apr 16 13:58 README.md
-rwxrwxr-x. 1 root root  1422 Apr 16 13:58 compose.yml
drwxrwxr-x. 1 root root     6 Apr 16 13:59 models
drwxrwxr-x. 1 root root    72 Apr 16 13:58 ollama
-rwxrwxr-x. 1 root root 22645 Apr 16 13:58 s-pilot
-rwxrwxr-x. 1 root root  1793 Apr 16 13:58 spilot_common.sh
-rwxrwxr-x. 1 root root  5919 Apr 16 13:58 spilot_install.sh
bash-4.4$ ./s-pilot
Welcome to Shell Pilot!!
You can quit with 'q' or 'e'.

<<You>>
hi


<<ShellPilot>>

<<You>>

reid41 avatar Apr 16 '24 14:04 reid41