docker-compose rwkv method folder structure

[bennmann]

looks like some filename variables may not be gracefully accounting for the example/rwkv folder structure now after #255 ? this is closer though (using the docker-compose method)

$ docker-compose up -d --build
....
....
#0 6.402 CMake Error: The source "/build/go-bert/bert.cpp/CMakeLists.txt" does not match the source "/media/username/LocalAI/go-bert/bert.cpp/CMakeLists.txt" used to generate cache.  Re-run cmake with a different source directory.
#0 6.403 make[1]: *** [Makefile:150: bert.o] Error 1
#0 6.403 make[1]: Leaving directory '/build/go-bert'
#0 6.403 make: *** [Makefile:94: go-bert/libgobert.a] Error 2
------
Dockerfile.dev:9
--------------------
   7 |     RUN apt-get update && apt-get install -y cmake
   8 |     COPY . .
   9 | >>> RUN make build
  10 |     
  11 |     FROM debian:$DEBIAN_VERSION
--------------------
ERROR: failed to solve: process "/bin/sh -c make build" did not complete successfully: exit code: 2

anyone else?

[Aisuko]

Hi, @bennmann. Thanks for your feedback. We will check it later.

[losemy]

+1

[bennmann]

+1

Please actually add a thumbs up to the original post via the little special button too! it helps track how many people have this issue

[bennmann]

fresh git clone from today:

`/LocalAI/examples/rwkv$ docker-compose up -d --build /usr/lib/python3/dist-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated "class": algorithms.Blowfish, Building api [+] Building 0.6s (15/15) FINISHED
=> [internal] load .dockerignore 0.1s => => transferring context: 114B 0.0s => [internal] load build definition from Dockerfile.dev 0.0s => => transferring dockerfile: 564B 0.0s => [internal] load metadata for docker.io/library/debian:11 0.4s => [internal] load metadata for docker.io/library/golang:1.20 0.5s => [builder 1/6] FROM docker.io/library/golang:1.20@sha256:685a22e459f9516f27d975c5cc6accc11223ee81fdfbbae60e39cc3b87357306 0.0s => [stage-1 1/3] FROM docker.io/library/debian:11@sha256:63d62ae233b588d6b426b7b072d79d1306bfd02a72bff1fc045b8511cc89ee09 0.0s => [internal] load build context 0.1s => => transferring context: 8.88kB 0.1s => CACHED [builder 2/6] WORKDIR /build 0.0s => CACHED [builder 3/6] RUN apt-get update && apt-get install -y cmake libgomp1 libopenblas-dev libopenblas-base libopencv-dev 0.0s => CACHED [builder 4/6] RUN ln -s /usr/include/opencv4/opencv2/ /usr/include/opencv2 0.0s => CACHED [builder 5/6] COPY . . 0.0s => CACHED [builder 6/6] RUN make build 0.0s => CACHED [stage-1 2/3] COPY --from=builder /build/local-ai /usr/bin/local-ai 0.0s => CACHED [stage-1 3/3] RUN apt-get update && apt-get install -y ca-certificates 0.0s => exporting to image 0.0s => => exporting layers 0.0s => => writing image sha256:f94fb52b3f55bb7e1a7a58fe7e10ab0ab935aea6200bec747e7c19280faf9e8c 0.0s => => naming to quay.io/go-skynet/local-ai:latest 0.0s Starting rwkv_api_1 ... done

rwkv$ curl http://localhost:8080/v1/completions -H "Content-Type: application/json" -d '{ "prompt": "A long time ago in a galaxy far, far away", "temperature": 0.7 }' {"error":{"code":500,"message":"could not load model - all backends returned error: 12 errors occurred:\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed rwkv$ curl http://localhost:8080/v1/models {"object":"list","data":[{"id":"RWKV-14B-11x-Q5_1.bin","object":"model"},{"id":"rwkv.tokenizer.json","object":"model"},{"id":"gpt-3.5-turbo","object":"model"}]}` I have also tried specifying the model.... I see in #301 somehow rwkv worked for someone, did they build locally instead of docker-compose method.... not sure why the backend is not loading the rwkv model this time, any insight?

[mudler]

fresh git clone from today:

`/LocalAI/examples/rwkv$ docker-compose up -d --build /usr/lib/python3/dist-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated "class": algorithms.Blowfish, Building api [+] Building 0.6s (15/15) FINISHED => [internal] load .dockerignore 0.1s => => transferring context: 114B 0.0s => [internal] load build definition from Dockerfile.dev 0.0s => => transferring dockerfile: 564B 0.0s => [internal] load metadata for docker.io/library/debian:11 0.4s => [internal] load metadata for docker.io/library/golang:1.20 0.5s => [builder 1/6] FROM docker.io/library/golang:1.20@sha256:685a22e459f9516f27d975c5cc6accc11223ee81fdfbbae60e39cc3b87357306 0.0s => [stage-1 1/3] FROM docker.io/library/debian:11@sha256:63d62ae233b588d6b426b7b072d79d1306bfd02a72bff1fc045b8511cc89ee09 0.0s => [internal] load build context 0.1s => => transferring context: 8.88kB 0.1s => CACHED [builder 2/6] WORKDIR /build 0.0s => CACHED [builder 3/6] RUN apt-get update && apt-get install -y cmake libgomp1 libopenblas-dev libopenblas-base libopencv-dev 0.0s => CACHED [builder 4/6] RUN ln -s /usr/include/opencv4/opencv2/ /usr/include/opencv2 0.0s => CACHED [builder 5/6] COPY . . 0.0s => CACHED [builder 6/6] RUN make build 0.0s => CACHED [stage-1 2/3] COPY --from=builder /build/local-ai /usr/bin/local-ai 0.0s => CACHED [stage-1 3/3] RUN apt-get update && apt-get install -y ca-certificates 0.0s => exporting to image 0.0s => => exporting layers 0.0s => => writing image sha256:f94fb52b3f55bb7e1a7a58fe7e10ab0ab935aea6200bec747e7c19280faf9e8c 0.0s => => naming to quay.io/go-skynet/local-ai:latest 0.0s Starting rwkv_api_1 ... done

rwkv$ curl http://localhost:8080/v1/completions -H "Content-Type: application/json" -d '{ "prompt": "A long time ago in a galaxy far, far away", "temperature": 0.7 }' {"error":{"code":500,"message":"could not load model - all backends returned error: 12 errors occurred:\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed loading model\n\t* failed rwkv$ curl http://localhost:8080/v1/models {"object":"list","data":[{"id":"RWKV-14B-11x-Q5_1.bin","object":"model"},{"id":"rwkv.tokenizer.json","object":"model"},{"id":"gpt-3.5-turbo","object":"model"}]}` I have also tried specifying the model.... I see in #301 somehow rwkv worked for someone, did they build locally instead of docker-compose method.... not sure why the backend is not loading the rwkv model this time, any insight?

The tokenizer file must be named after the model:

model: RWKV-14B-11x-Q5_1.bin tokenizer: RWKV-14B-11x-Q5_1.bin.tokenizer.json

I'd suggest to use the model gallery to install rwkv https://github.com/go-skynet/model-gallery#gpt-rwkv-7b:

LOCALAI=http://localhost:8080
curl $LOCALAI/models/apply -H "Content-Type: application/json" -d '{
     "url": "github:go-skynet/model-gallery/rwkv-raven-7b.yaml",
     "name": "rwkv"
   }'  

[localai-bot]

:warning::warning::warning::warning::warning:

Hi! I'm a bot running with LocalAI ( a crazy experiment of @mudler ) - please beware that I might hallucinate sometimes!

but.... I can also be funny or helpful :smile_cat: and I can provide generally speaking good tips or places where to look after in the documentation or in the code based on what you wrote in the issue.

Don't engage in conversation with me, I don't support (yet) replying!

:warning::warning::warning::warning::warning:

It seems like the issue is related to the CMake configuration. The error message indicates that the source file path in the CMakeLists.txt file does not match the actual source file path on the system. This could be due to a misconfiguration in the CMake setup or a mismatch between the expected and actual file structure on the system. To troubleshoot this issue, you can try the following steps:

1. Check the CMake configuration files for any errors or inconsistencies. Ensure that the source file paths are correctly specified and match the actual file structure on the system.
2. Verify that the CMake installation is up to date and compatible with the Go build toolchain. You may need to update the CMake version or install additional dependencies if necessary.
3. Check the Docker build environment configuration to ensure that it matches the expected setup for the project. This includes verifying the correct use of volumes, directories, and other settings.
4. If the issue persists after these steps, you may need to seek assistance from the project maintainers or consult the CMake documentation for further troubleshooting.

Sources:

• https://github.com/go-skynet/LocalAI/tree/master/Makefile
• https://github.com/go-skynet/LocalAI/tree/master/.vscode/launch.json
• https://github.com/go-skynet/LocalAI/tree/master/entrypoint.sh

[Aisuko]

Hi guys, I am going to close this issue because there is no more response. If you still get stuck in this, please reopen it.