containers icon indicating copy to clipboard operation
containers copied to clipboard

[bitnami/minio] Connections failing with minio client in Python.

Open katyakinane opened this issue 7 months ago • 6 comments

Name and Version

[bitnami/minio:2025]

What architecture are you using?

amd64

What steps will reproduce the bug?

We run github actions that require a MinIO instance. Our configuration has the following services that are most relevant to this:

name: Tests

on:
  push:
    branches: [ 'main', 'master', 'develop' ]
  pull_request:
    branches: [ '**' ]

jobs:
  code-quality:
    runs-on: ubuntu-latest
    env:
      CELERY_BROKER_URL: redis://localhost:6379/0
      CELERY_RESULT_BACKEND: redis://localhost:6379/0
      REDIS_HOST: localhost
      REDIS_PORT: 6379
      REDIS_DB: 0
      MINIO_ENDPOINT: localhost:9000
      MINIO_ACCESS_KEY: minioadmin
      MINIO_SECRET_KEY: minioadmin
      MINIO_BUCKET_NAME: nettle-files-test
      MINIO_SECURE: false
    services:
      redis:
        image: redis:6
        ports:
          - 6379:6379
        options: >-
          --health-cmd "redis-cli ping"
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

      minio:
        image: bitnami/minio:latest
        ports:
          - 9000:9000
        env:
          MINIO_ROOT_USER: minioadmin
          MINIO_ROOT_PASSWORD: minioadmin
        options: >-
          --health-cmd "curl -f http://localhost:9000/minio/health/ready"
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

    steps:
    - uses: actions/checkout@v3

    - name: Set up Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.11'
        cache: 'pip'

    - name: Set up System
      run: |
        sudo apt-get update
        sudo apt-get install -y weasyprint

    - name: Install Poetry
      run: |
        pip install minio

    - name: Run test setup
      run: |
        python test_setup.py

We then run a setup script to connect to the MinIO instance before running the tests which as of the latest update, fails.

# test_setup.py
from minio import Minio
import os

# adapted from a BaseCommand in Django for easier replication
class SetUp:
    def handle(self, *args, **options):
        self.setup_minio()
        print("SUCCESS" + "Test environment setup complete!")

    def setup_minio(self):
        """Create MinIO bucket for tests if it doesn't exist."""
        print("Setting up MinIO storage...")
        try:
            client = Minio(
                os.environ.get('MINIO_ENDPOINT'),
                access_key=os.environ.get('MINIO_ACCESS_KEY'),
                secret_key=os.environ.get('MINIO_SECRET_KEY'),
                secure=bool(os.environ.get('MINIO_SECURE', 'false').lower() == 'true'),
            )

            bucket_name = os.environ.get('MINIO_BUCKET_NAME')
            if not client.bucket_exists(bucket_name):
                client.make_bucket(bucket_name)
                print("SUCCESS" + f"Created MinIO bucket: {bucket_name}")
            else:
                print(f"MinIO bucket already exists: {bucket_name}")

            return True
        except Exception as e:
            print("ERROR" + f"MinIO setup failed: {str(e)}")
            return False

if __name__ == '__main__':
    SetUp().handle()


We run this command as part of our Django setup normally as a BaseCommand but it has been adapted for ease of replication.

What is the expected behavior?

Yesterday this step passed:

Run poetry run python manage.py setup_test_storage
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): localhost:9000
DEBUG:urllib3.connectionpool:http://localhost:9000 "GET /nettle-files-test?location= HTTP/1.1" 404 0
DEBUG:urllib3.connectionpool:http://localhost:9000 "PUT /nettle-files-test HTTP/1.1" 200 0
Setting up MinIO storage...
Created MinIO bucket: nettle-files-test

What do you see instead?

Run poetry run python manage.py setup_test_storage
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): localhost:9000
DEBUG:urllib3.util.retry:Incremented Retry for (url='/nettle-files-test?location='): Retry(total=4, connect=None, read=None, redirect=None, status=None)
WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5810[30](https://github.com/Nettle-AI/mvp/actions/runs/15328986710/job/43130744138#step:12:31)c8d0>: Failed to establish a new connection: [Errno 111] Connection refused')': /nettle-files-test?location=
DEBUG:urllib3.connectionpool:Starting new HTTP connection (2): localhost:9000
DEBUG:urllib3.util.retry:Incremented Retry for (url='/nettle-files-test?location='): Retry(total=3, connect=None, read=None, redirect=None, status=None)
WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f581033ac90>: Failed to establish a new connection: [Errno 111] Connection refused')': /nettle-files-test?location=
DEBUG:urllib3.connectionpool:Starting new HTTP connection (3): localhost:9000
DEBUG:urllib3.util.retry:Incremented Retry for (url='/nettle-files-test?location='): Retry(total=2, connect=None, read=None, redirect=None, status=None)
WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5810[33](https://github.com/Nettle-AI/mvp/actions/runs/15328986710/job/43130744138#step:12:34)b810>: Failed to establish a new connection: [Errno 111] Connection refused')': /nettle-files-test?location=
DEBUG:urllib3.connectionpool:Starting new HTTP connection (4): localhost:9000
DEBUG:urllib3.util.retry:Incremented Retry for (url='/nettle-files-test?location='): Retry(total=1, connect=None, read=None, redirect=None, status=None)
WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5810[34](https://github.com/Nettle-AI/mvp/actions/runs/15328986710/job/43130744138#step:12:35)4410>: Failed to establish a new connection: [Errno 111] Connection refused')': /nettle-files-test?location=
DEBUG:urllib3.connectionpool:Starting new HTTP connection (5): localhost:9000
DEBUG:urllib3.util.retry:Incremented Retry for (url='/nettle-files-test?location='): Retry(total=0, connect=None, read=None, redirect=None, status=None)
WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5810344f90>: Failed to establish a new connection: [Errno 111] Connection refused')': /nettle-files-test?location=
DEBUG:urllib3.connectionpool:Starting new HTTP connection (6): localhost:9000
Setting up MinIO storage...
MinIO setup failed: HTTPConnectionPool(host='localhost', port=9000): Max retries exceeded with url: /nettle-files-test?location= (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5810345b10>: Failed to establish a new connection: [Errno 111] Connection refused'))

Additional information

Minio Version:

minio = "^7.2.9"

katyakinane avatar May 29 '25 17:05 katyakinane

Same here connecting through boto3 library

botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://localhost:9000/...

We've pinned to 2025.4.22 instead of latest as workaround.

rowanseymour avatar May 29 '25 17:05 rowanseymour

I've struggled to recreate the issue in a separate action so it seems to be an interaction with django and other dependencies.

katyakinane avatar May 29 '25 18:05 katyakinane

It seems it's the container failing to start, so not the client at all: https://github.com/bitnami/containers/issues/81607

rowanseymour avatar May 29 '25 18:05 rowanseymour

Same here connecting through boto3 library

botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://localhost:9000/...

We've pinned to 2025.4.22 instead of latest as workaround.

The workaround worked for me. Thank you ! Was looking for any urllib3 or boto3 errors but I think you're right about the container

gsysaath avatar Jun 02 '25 09:06 gsysaath

Yet again, the latest Docker image fails to perform correctly. Pinned to 2025.4.8

devrivne avatar Jun 03 '25 05:06 devrivne

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

github-actions[bot] avatar Jun 19 '25 01:06 github-actions[bot]

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

github-actions[bot] avatar Jul 06 '25 01:07 github-actions[bot]

Due to the lack of activity in the last 5 days since it was marked as "stale", we proceed to close this Issue. Do not hesitate to reopen it later if necessary.

github-actions[bot] avatar Jul 11 '25 01:07 github-actions[bot]