Upload error when running in docker localhost

[a0v0]

docker-compose.yml

services:
  minio:
    container_name: Minio
    image: minio/minio
    command: server /mnt/data --console-address ":9001"
    ports:
      - 9000:9000 # API
      - 9001:9001 # Webinterface
    environment:
      MINIO_ROOT_USER: "minio-admin" # changeme!
      MINIO_ROOT_PASSWORD: "minio-pass" # changeme!
      MINIO_SERVER_URL: "http://localhost:9000"
      MINIO_SITE_REGION: "s3"
    volumes:
      - minio-data:/mnt/data
    healthcheck:
      test:
        [
          "CMD",
          "curl",
          "-f",
          "http://localhost:9000/minio/health/ready"
        ]
      interval: 30s
      timeout: 20s
      retries: 3

  gose:
    image: ghcr.io/stv0g/gose:v0.6.0
    ports:
      - 8080:8080
    env_file: .env
    depends_on:
      - minio

.env

# Listen address and port of Gose
GOSE_LISTEN= ":8080"
# Base URL at which Gose is accessible
GOSE_BASE_URL= "http://localhost:8080"
# Name of S3 bucket
GOSE_BUCKET= "gose-uploads"
# Hostname:Port of S3 server
GOSE_ENDPOINT= "minio:9000"
# Region of S3 server
GOSE_REGION= "s3"
# Prepend bucket name to path
GOSE_PATH_STYLE= "true"
# Disable SSL encryption for S3
GOSE_NO_SSL= "true"
# S3 Access Key
GOSE_ACCESS_KEY= "minio-admin"
# S3 Secret Key 
GOSE_SECRET_KEY= "minio-pass"
# Create S3 bucket if do not exists
GOSE_SETUP_BUCKET= "true"
# Setup S3 bucket CORS rules
GOSE_SETUP_CORS="true"
# Setup S3 bucket lifecycle rules
GOSE_SETUP_LIFECYCLE="true"
# Maximum upload size
GOSE_MAX_UPLOAD_SIZE= "50GB"
# Part-size for multi-part uploads
GOSE_PART_SIZE= "16MB"
# Number of days after which incomplete uploads are cleaned-up (set to 0 to disable)
GOSE_SETUP_ABORT_INCOMPLETE_UPLOADS=31

[a0v0]

also can gose be hosted as an API only server without frontend?

[stv0g]

also can gose be hosted as an API only server without frontend?

In principle yes, however the API is slightly more elaborate due to the chunking and multi-part uploads.

However, I have a Golang CLI client which implements this API on my roadmap. Contributions are also highly welcome :)

[stv0g]

Regarding the docker-compose setup:

Can you try changing GOSE_ENDPOINT to localhost:9000?

We do not yet support internal and public S3 endpoints for the servers. Right now we are expecting that endpoint is both reachable by the backend as well as the clients as they upload their files directly to S3 rather than proxying the uploads through the gose backend.

[a0v0]

Regarding the docker-compose setup:

Can you try changing GOSE_ENDPOINT to localhost:9000?

We do not yet support internal and public S3 endpoints for the servers. Right now we are expecting that endpoint is both reachable by the backend as well as the clients as they upload their files directly to S3 rather than proxying the uploads through the gose backend.

Ok I will try this

[fdterr]

I am also having this issue when using GOSE_ENDPOINT: "minio:9000"

If I set GOSE_ENDPOINT to localhost:9000 then I get the following error after running docker-compose up. My docker-compose file is below.

gose_1   | 2022/09/03 06:46:02 Detected Unknown S3 implementation for server http://localhost:9000/gose-uploads
gose_1   | 2022/09/03 06:46:03 Failed to setup servers: failed to set bucket gose-uploads's CORS rules: RequestError: send request failed
gose_1   | caused by: Put "http://localhost:9000/gose-uploads?cors=": dial tcp 127.0.0.1:9000: connect: connection refused

---
version: "3.7"
services:

  minio:
    image: minio/minio:RELEASE.2022-06-03T01-40-53Z.fips
    command: server /mnt/data --console-address ":9001"
    ports:
    - 9000:9000 # API
    - 9001:9001 # Webinterface
    environment:
      MINIO_ROOT_USER: "admin-user" # changeme!
      MINIO_ROOT_PASSWORD: "admin-pass"  # changeme!
      MINIO_SERVER_URL: "http://127.0.0.1:9000"
      MINIO_SITE_REGION: "us-east-1"
    volumes:
    - vol1:/mnt/data
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/ready"]
      interval: 30s
      timeout: 20s
      retries: 3

  gose:
    image: ghcr.io/stv0g/gose:v0.4.0
    build:
      context: .
    ports:
    - 8080:8080
    environment:
      GOSE_LISTEN: ":8080"
      GOSE_BASE_URL: "http://localhost:8080"
      GOSE_BUCKET: "gose-uploads"
      GOSE_ENDPOINT: "minio:9000"
      GOSE_REGION: "us-east-1"
      GOSE_PATH_STYLE: "false"
      GOSE_NO_SSL: "true"
      GOSE_SETUP_BUCKET: "true"
      GOSE_ACCESS_KEY: "xxxx" # changeme!
      GOSE_SECRET_KEY: "xxxxxx" # changeme!
      GOSE_MAX_UPLOAD_SIZE: "50GB"
      GOSE_PART_SIZE: "16MB"
      GOSE_SETUP_CORS: "true"
      GOSE_SETUP_LIFECYCLE: "false"
    depends_on:
    - minio

volumes:
  vol1:
    external

[stv0g]

Hi @fdterr,

I did some more research on this issue. And it seems like there is no easy solution. As I said before, the long-term solution could be that we enhance Gose to support and internal and external S3 endpoint URL.

Since the localhost workaround does not work, could you maybe try to use a local IP address of the machine on which docker-compose is running instead?

[fdterr]

I just tried using the local IP address of the machine as well as host.docker.internal as the IP address and adding the line below to the gose container definition in docker-compose:

extra_hosts:
    - "host.docker.internal:host-gateway"

Have you been able to get docker-compose working yourself? @stv0g Or do you now is there some further configuration we have to do on minio to allow external access to the API?

EDIT: when I add host.docker.internal:9000 as the GOSE_ENDPOINT and I enter the gose container and try wget host.docker.internal:9000, I get the following error: wget: server returned error: HTTP/1.1 403 Forbidden

EDIT EDIT: When I add a useragent, I am able to connect to minio from the gose container:

/ # wget host.docker.internal:9000 -U "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Sa
fari/537.36"
Connecting to host.docker.internal:9000 (172.17.0.1:9000)
Connecting to host.docker.internal:9001 (172.17.0.1:9001)
saving to 'index.html'
index.html           100% |****************************************************************************************************************|  1266  0:00:00 ETA
'index.html' saved