Does not work with custom endpoint_url & region_name

[M0dM]

Hi,

I am trying to make KEV work with minio: https://github.com/minio/minio I got this error:

    File ~/.local/share/virtualenvs/myvenv/lib/python3.8/site-packages/kev/backends/s3/db.py", line 25, in __init__
        boto3.Session(**session_kwargs)
    TypeError: __init__() got an unexpected keyword argument 'endpoint_url'

My config is this one :

    's3': {
        'backend': 'kev.backends.s3.db.S3DB',
        'connection': {
            'bucket': 'mybucketname',
            'endpoint_url': 'http://localhost:9000'  # The port of my minio local deployment
        }
    }

I had dig a little bit inside the code and found the issue: As described here : https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html#boto3.session.Session

boto3.Session reference attributes (no endpoint_url):

    class boto3.session.Session(aws_access_key_id=None, aws_secret_access_key=None, aws_session_token=None, region_name=None, botocore_session=None, profile_name=None)

Boto3 Session does not support endpoint_url argument. This argument must be added when creating s3 boto resource.

What could be done is something like this in kev.backends.s3.db.S3DB class constructor:

    def __init__(self,**kwargs):
        #
        session_kwargs = {k: v for k, v in kwargs.items() if k in
                          self.session_kwargs}
        endpoint_url = None
        if 'endpoint_url' in session_kwargs.keys():
            endpoint_url =  session_kwargs['endpoint_url']   
            del(session_kwargs['endpoint_url'])
        if len(session_kwargs.keys()) > 0:
            boto3.Session(**session_kwargs)

        self._db = boto3.resource('s3', endpoint_url=endpoint_url)
        self.bucket = kwargs['bucket']
        self._indexer = self._db.Bucket(self.bucket)

boto3.resource reference attributes (existing endpoint_url kwarg):

    resource(service_name, region_name=None, api_version=None, use_ssl=True, verify=None, endpoint_url=None, aws_access_key_id=None, aws_secret_access_key=None, aws_session_token=None, config=None)[source]

endpoint_url default value is None, then this code should always be ok.

What do you think about this fix / workaround ?

Edit: We should add support to region_name also...

[bjinwright]

Endpoint URL is supported as of 0.10