localstack-python-client icon indicating copy to clipboard operation
localstack-python-client copied to clipboard

[Feature] Support virtual host S3 API calls

Open mfontanaar opened this issue 1 year ago • 0 comments

After struggling for several hours with Docker Compose, I noticed my problem was actually in this library, which does not seem to support the virtual host addressing for the s3 client.

Looking at boto3 in debug mode, I noticed that when using Virtual host addressing, the HTTP request was aimed towards http://<bucket>.<endpoint_url>:<endpoint_port>. This is not a valid endpoint, and it should be http://<bucket>.s3.<endpoint_url>:<endpoint_port>, as described here.

After delving deeper I noticed localstack_client.config.get_endpoint it is not handling s3 in any special way. I manually patched the function and noticed that with a simple

def new_get_service_endpoint(
        service: str, localstack_host: Optional[str] = None
    ) -> Optional[str]:
        endpoints = localstack_client.config.get_service_endpoints(localstack_host=localstack_host)
        endpoint = endpoints.get(service)
        if service == "s3":
            endpoint = "http://s3." + endpoint.split("http://")[1]
        return endpoint

I know this doesn't handle SSL, it's just a draft. It works for both virtual and path based addressing.

I see three ways simple about this:

  1. Do not support virtual host addressing. This is a problem, as in theory AWS is deprecating path based addressing (also it has been deprecating it for 4 years, so...)
  2. Do a hack like the one proposed.
  3. Honor AWS service specific endpoints through envars (for instance AWS_ENDPOINT_URL_S3) and leave it up to the user to set it up (with appropriate documentation, at least in the README.md).

What are your thoughts on this?

mfontanaar avatar Mar 21 '24 15:03 mfontanaar