coolify icon indicating copy to clipboard operation
coolify copied to clipboard

[Bug]: Can't reach Minio S3 console after deploy

Open aybasaran opened this issue 9 months ago • 19 comments

Description

Hello Everyone, I've been facing problems with MinioS3. I can't react the minio console after deploy.

Minimal Reproduction (if possible, example repository)

Exception or Error

No response

Version

v4.0.0-beta.271

aybasaran avatar May 03 '24 01:05 aybasaran

how did you managed to fix this issue?

nimaa77 avatar May 08 '24 11:05 nimaa77

how did you managed to fix this issue?

i didn't. I think its a major problem. I guess, we should use v3 or use custom docker-compose.yml.

aybasaran avatar May 08 '24 12:05 aybasaran

@nimaa77 i found a solution. I change my proxy server into Caddy and it worked

aybasaran avatar May 14 '24 07:05 aybasaran

this problem still persists. I tried to debug the docker compose by myself, but with no luck. Traefik rule doesn't appear on dashboard, so it's probably related to that.

nunocasteleira avatar May 16 '24 18:05 nunocasteleira

this problem still persists. I tried to debug the docker compose by myself, but with no luck. Traefik rule doesn't appear on dashboard, so it's probably related to that.

Did you try using Caddy as a proxy server? That's an easy process. image

aybasaran avatar May 16 '24 22:05 aybasaran

this problem still persists. I tried to debug the docker compose by myself, but with no luck. Traefik rule doesn't appear on dashboard, so it's probably related to that.

Did you try using Caddy as a proxy server? That's an easy process.

image

Yes, but I prefer to keep traefik. Nonetheless, I'll take a look on using Caddy.

nunocasteleira avatar May 16 '24 22:05 nunocasteleira

I'm using Caddy but still cannot login. Any other ideas?

nicnocquee avatar Jun 11 '24 13:06 nicnocquee

I'm using Caddy but still cannot login. Any other ideas?

Did you add DNS records?

image

aybasaran avatar Jun 11 '24 13:06 aybasaran

i don't use cloudflare, in namecheap like this:

image

and in coolify

Screenshot 2024-06-11 at 15 51 28

version v4.0.0-beta.294 btw

nicnocquee avatar Jun 11 '24 13:06 nicnocquee

i don't use cloudflare, in namecheap like this:

image

and in coolify

Screenshot 2024-06-11 at 15 51 28 version v4.0.0-beta.294 btw

Can you try uploads.example.com and minio-console.example.com instead of *.minio.example.com. (aliases up to you, you can try what ever you want but not double sub domain)

aybasaran avatar Jun 11 '24 14:06 aybasaran

Still doesn't work. I noticed when I visited the console URL, it always took a while to redirect to login. When I looked at the network log, there was a timeout calling https://minio.my-domain/api/v1/session with error:

{
    "detailedMessage": "Get \"https://upload-minio.my-domain/minio/admin/v3/accountinfo\": dial tcp <ip-address>:443: i/o timeout",
    "message": "invalid session"
}

i wonder if it's related.

some more info if needed:

  • the server running minio is not the same with coolify instance.
  • i have entered the wildcard domain in the server -> general setting page
  • i don't change anything in the caddy configuration file.

nicnocquee avatar Jun 11 '24 14:06 nicnocquee

So i tried to add a new server, then installed and deployed minio, without any change, no custom domain. It worked I could login but no https.

Then I set my custom domain with https in the minio setting. This caused the "Invalid Login" error.

Then I changed the proxy to caddy. Now it worked!

So I guess because in my previous server I have the wildcard domain set? Any ideas how to fix without removing the wildcard domain?

nicnocquee avatar Jun 11 '24 15:06 nicnocquee

So i tried to add a new server, then installed and deployed minio, without any change, no custom domain. It worked I could login but no https.

Then I set my custom domain with https in the minio setting. This caused the "Invalid Login" error.

Then I changed the proxy to caddy. Now it worked!

So I guess because in my previous server I have the wildcard domain set? Any ideas how to fix without removing the wildcard domain?

Double subdomain is something new for coolify i guess. Just use single subdomain with the wildcard. I always use this.

aybasaran avatar Jun 11 '24 15:06 aybasaran

I think the difference between your setup and mine is that I have two servers, one for coolify instance and another one where I tried to deploy minio.

The coolify instance has https://coolify.example.com domain with Traefik proxy. The other server where I installed minio has https://magic.coolify.example.com wildcard with Caddy proxy.

When I added the third server without setting the wildcard domain (but with caddy proxy), minio can be accessed.

Honestly I'm not familiar with Traefik and Caddy but that's my finding.

nicnocquee avatar Jun 11 '24 15:06 nicnocquee

After tinkering for hours and a good quick nap, I finally figured out the problem: I have ufw enabled in my server!

Disabling the ufw fixed it. But I want to keep the ufw enabled. Allowing incoming to port 9000 and 9001 didn't work. What worked was adding rule to allow all incoming traffic from the ip address of the minio container:

  1. Get the ip of the minio container: sudo docker network inspect <minio service uuid>. You can get the uuid from the url in the coolify dashboard when you're in the minio service page, like https://coolifydomain/project/<project-id>/production/service/<the-minio-id-we-want>#webhooks
  2. Find the "IPv4Address" of the container in the output of the command above.
  3. then run sudo ufw allow from <the ipv4 address from step 2>
  4. sudo ufw reload

nicnocquee avatar Jun 12 '24 00:06 nicnocquee

Same here. I have made multiple attempts with minio. The containers are able to start, and they run in an unhealthy state. When going to the console URL, I always get 404.

exoer avatar Jun 28 '24 07:06 exoer

I am facing the same difficulties here, too: an unhealthy state of Minio. If I change the proxy from Traefik to Caddy, Minio works, but one of our services, Ghost, stops working. Well, I would rather stay with Traefik for now. Version: v4.0.0-beta.306

tolstenko avatar Jun 28 '24 14:06 tolstenko

Just to add the screenshots of the steps to reproduce on the latest version 306 and Traefik. It works fine with Caddy, but if i use Caddy, Ghost stops working.

Details

0 1 2 3 4 5

tolstenko avatar Jun 28 '24 16:06 tolstenko

@tolstenko So it looks like the problem is related to the proxy 🧐

exoer avatar Jun 28 '24 17:06 exoer

This problem has already been resolved in PR #2829

mateusfmello avatar Jul 15 '24 13:07 mateusfmello