LocalAI icon indicating copy to clipboard operation
LocalAI copied to clipboard
Published 2 weeks ago • verified publisher icon mudler

How to compile LocalAI with stablediffusion on Docker container

Open Soberia opened this issue 2 years ago • 16 comments

LocalAI version: v1.13.0 commit ffaf3b1d362cc4d0045e1ed60bbcf0481d21f8ef

Describe the bug I changed make build to make GO_TAGS=stablediffusion build in Dockerfile and during the build process, I can see in the logs that the github.com/mudler/go-stable-diffusion repository gets cloned but whenever I make an API call to the /v1/images/generations endpoint, I always get this error:

This version of LocalAI was built without the stablediffusion tag

directory content:

root@ai:/home/ai/LocalAI/models/stablediffusion_assets# du -ahd1
95M	./AutoencoderKL-fp16.bin
60K	./UNetModel-256-MHA-fp16.param
470M	./FrozenCLIPEmbedder-fp32.onnx
190M	./AutoencoderKL-fp32.onnx
1.7G	./UNetModel-MHA-fp16.bin
104K	./UNetModel-fp16.param
36K	./FrozenCLIPEmbedder-fp16.param
66M	./AutoencoderKL-encoder-512-512-fp16.bin
20K	./AutoencoderKL-512-fp16-opt.param
236M	./FrozenCLIPEmbedder-fp16.bin
76K	./UNetModel-512-MHA-fp16-opt.param
56K	./UNetModel-512-MHA-fp16.param
16K	./AutoencoderKL-512-fp16.param
20K	./AutoencoderKL-256-fp16-opt.param
16K	./AutoencoderKL-256-fp16.param
1.7G	./UNetModel-fp16.onnx
76K	./UNetModel-256-MHA-fp16-opt.param

Soberia avatar May 20 '23 00:05 Soberia

https://github.com/go-skynet/LocalAI#build-with-image-generation-support

but i donj't know how to run this command.

ralyodio avatar May 20 '23 08:05 ralyodio

I think you have to change BUILD_TYPE in .env but i'm not sure to what.

ralyodio avatar May 20 '23 08:05 ralyodio

I just tried what you did originally and i canj't connect:


$ curl http://localhost:9220/v1/models
curl: (7) Failed to connect to localhost port 9220 after 0 ms: Couldn't connect to server

ralyodio avatar May 20 '23 09:05 ralyodio

I think you have to change BUILD_TYPE in .env but i'm not sure to what.

I tried with BUILD_TYPE=openblas before, no luck either.

Soberia avatar May 20 '23 10:05 Soberia

This is a lack in our docs. Did you tried by setting GO_TAGS=stablediffusion in the .env file?

mudler avatar May 20 '23 12:05 mudler

i still get {"error":{"code":500,"message":"This version of LocalAI was built without the stablediffusion tag","type":""}}

after add GO_TAGS to .env

ralyodio avatar May 20 '23 12:05 ralyodio

i still get {"error":{"code":500,"message":"This version of LocalAI was built without the stablediffusion tag","type":""}}

after add GO_TAGS to .env

How are you running LocalAI? Do you set also REBUILD=false by any chance? default binaries in the image don't have stable diffusion enabled, but I think it's safe to enable at this point.

mudler avatar May 20 '23 13:05 mudler

no i have REBUILD=true

# THREADS=14
# CONTEXT_SIZE=512
MODELS_PATH=/models
DEBUG=true
#BUILD_TYPE=generic
REBUILD=true
PORT=9220
GO_TAGS=stablediffusion

ralyodio avatar May 20 '23 13:05 ralyodio

I commented if statement in Makefile to make sure required packages are compiled at the build time. I think this error is not related to the presence of stablediffusion. I don't know the Go language but is this function throws this error for any failure?

Soberia avatar May 20 '23 16:05 Soberia

Did you end up getting this working? Trying to figure out how to do this for kubernetes

Edit: Adding GO_TAGS: stablediffiusion to the container environment variables puts the container into a crash loop

lenaxia avatar May 22 '23 03:05 lenaxia

Did you end up getting this working? Trying to figure out how to do this for kubernetes

Edit: Adding GO_TAGS: stablediffiusion to the container environment variables puts the container into a crash loop

can you take the logs and file a separate issue for this? I'm running this in Kubernetes and don't observe any crashloop here

mudler avatar May 22 '23 20:05 mudler

no i have REBUILD=true

# THREADS=14
# CONTEXT_SIZE=512
MODELS_PATH=/models
DEBUG=true
#BUILD_TYPE=generic
REBUILD=true
PORT=9220
GO_TAGS=stablediffusion

Can you share how you start LocalAI and the full logs?

a make GO_TAGS=stablediffusion build here yields:

base ❯ make GO_TAGS=stablediffusion build                                                                                                                                                                                                    
[...]
I local-ai build info:
I BUILD_TYPE: 
I GO_TAGS: stablediffusion
CGO_LDFLAGS="" C_INCLUDE_PATH=/home/mudler/_git/LocalAI/go-llama:/home/mudler/_git/LocalAI/go-stable-diffusion/:/home/mudler/_git/LocalAI/gpt4all/gpt4all-bindings/golang/:/home/mudler/_git/LocalAI/go-gpt2:/home/mudler/_git/LocalAI/go-rwk
v:/home/mudler/_git/LocalAI/whisper.cpp:/home/mudler/_git/LocalAI/go-bert:/home/mudler/_git/LocalAI/bloomz LIBRARY_PATH=/home/mudler/_git/LocalAI/go-llama:/home/mudler/_git/LocalAI/go-stable-diffusion/:/home/mudler/_git/LocalAI/gpt4all/g
pt4all-bindings/golang/:/home/mudler/_git/LocalAI/go-gpt2:/home/mudler/_git/LocalAI/go-rwkv:/home/mudler/_git/LocalAI/whisper.cpp:/home/mudler/_git/LocalAI/go-bert:/home/mudler/_git/LocalAI/bloomz go build -tags "stablediffusion" -x -o l
ocal-ai ./
WORK=/tmp/go-build1217960430

mudler avatar May 22 '23 20:05 mudler

a make GO_TAGS=stablediffusion build here yields:

base ❯ make GO_TAGS=stablediffusion build                                                                                                                                                                                                    
[...]
I local-ai build info:
I BUILD_TYPE: 
I GO_TAGS: stablediffusion
CGO_LDFLAGS="" C_INCLUDE_PATH=/home/mudler/_git/LocalAI/go-llama:/home/mudler/_git/LocalAI/go-stable-diffusion/:/home/mudler/_git/LocalAI/gpt4all/gpt4all-bindings/golang/:/home/mudler/_git/LocalAI/go-gpt2:/home/mudler/_git/LocalAI/go-rwk
v:/home/mudler/_git/LocalAI/whisper.cpp:/home/mudler/_git/LocalAI/go-bert:/home/mudler/_git/LocalAI/bloomz LIBRARY_PATH=/home/mudler/_git/LocalAI/go-llama:/home/mudler/_git/LocalAI/go-stable-diffusion/:/home/mudler/_git/LocalAI/gpt4all/g
pt4all-bindings/golang/:/home/mudler/_git/LocalAI/go-gpt2:/home/mudler/_git/LocalAI/go-rwkv:/home/mudler/_git/LocalAI/whisper.cpp:/home/mudler/_git/LocalAI/go-bert:/home/mudler/_git/LocalAI/bloomz go build -tags "stablediffusion" -x -o l
ocal-ai ./
WORK=/tmp/go-build1217960430

@mudler Thanks for the hint. I think I found the problem. Here's the log when GO_TAGS=stablediffusion is set in .env:

Screenshot 2023-05-26 035753

When I change the Dockerfile.dev make command to:

sed -i 's/make build/make GO_TAGS=stablediffusion build/' Dockerfile.dev

eventually, stablediffusion will be compiled:

image

I also tried to modify the dockerfile manually before but I was looking for the wrong one. The Dockerfile.dev is referenced in compose file and not the Dockerfile. (why?)

So I think to fix this issue, in either Dockerfile or Dockerfile.dev, RUN make build should be changed to RUN make $GO_TAGS build to read the environment variable correctly for the build process.

Soberia avatar May 26 '23 03:05 Soberia

a make GO_TAGS=stablediffusion build here yields:

base ❯ make GO_TAGS=stablediffusion build                                                                                                                                                                                                    
[...]
I local-ai build info:
I BUILD_TYPE: 
I GO_TAGS: stablediffusion
CGO_LDFLAGS="" C_INCLUDE_PATH=/home/mudler/_git/LocalAI/go-llama:/home/mudler/_git/LocalAI/go-stable-diffusion/:/home/mudler/_git/LocalAI/gpt4all/gpt4all-bindings/golang/:/home/mudler/_git/LocalAI/go-gpt2:/home/mudler/_git/LocalAI/go-rwk
v:/home/mudler/_git/LocalAI/whisper.cpp:/home/mudler/_git/LocalAI/go-bert:/home/mudler/_git/LocalAI/bloomz LIBRARY_PATH=/home/mudler/_git/LocalAI/go-llama:/home/mudler/_git/LocalAI/go-stable-diffusion/:/home/mudler/_git/LocalAI/gpt4all/g
pt4all-bindings/golang/:/home/mudler/_git/LocalAI/go-gpt2:/home/mudler/_git/LocalAI/go-rwkv:/home/mudler/_git/LocalAI/whisper.cpp:/home/mudler/_git/LocalAI/go-bert:/home/mudler/_git/LocalAI/bloomz go build -tags "stablediffusion" -x -o l
ocal-ai ./
WORK=/tmp/go-build1217960430

@mudler Thanks for the hint. I think I found the problem. Here's the log when GO_TAGS=stablediffusion is set in .env:

Screenshot 2023-05-26 035753

When I change the Dockerfile.dev make command to:

sed -i 's/make build/make GO_TAGS=stablediffusion build/' Dockerfile.dev

eventually, stablediffusion will be compiled:

image

I also tried to modify the dockerfile manually before but I was looking for the wrong one. The Dockerfile.dev is referenced in compose file and not the Dockerfile. (why?)

So I think to fix this issue, in either Dockerfile or Dockerfile.dev, RUN make build should be changed to RUN make $GO_TAGS build to read the environment variable correctly for the build process.

how are you running it in docker-compose? try to pull the image and specify --pull always instead of --build. --build will use Dockerfile.dev which is used for development and doesn't recompile on start.

mudler avatar May 26 '23 07:05 mudler

how are you running it in docker-compose?

git clone https://github.com/go-skynet/LocalAI
cd LocalAI && docker compose build

Soberia avatar May 26 '23 12:05 Soberia

how are you running it in docker-compose?

git clone https://github.com/go-skynet/LocalAI
cd LocalAI && docker compose build

Try with:

git clone https://github.com/go-skynet/LocalAI
cd LocalAI && docker compose up --pull always

mudler avatar May 26 '23 16:05 mudler

@mudler The GO_TAGS environment variable is set correctly when the image builds with docker compose up instead of docker compose build and there is no need to modify the dockerfile manually.

Thank you. I close this now.

Soberia avatar May 27 '23 01:05 Soberia