Does not work with custom endpoint_url & region_name #42
Description
Hi,
I am trying to make KEV work with minio: https://github.com/minio/minio
I got this error:
File ~/.local/share/virtualenvs/myvenv/lib/python3.8/site-packages/kev/backends/s3/db.py", line 25, in __init__
boto3.Session(**session_kwargs)
TypeError: __init__() got an unexpected keyword argument 'endpoint_url'
My config is this one :
's3': {
'backend': 'kev.backends.s3.db.S3DB',
'connection': {
'bucket': 'mybucketname',
'endpoint_url': 'http://localhost:9000' # The port of my minio local deployment
}
}I had dig a little bit inside the code and found the issue:
As described here : https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html#boto3.session.Session
boto3.Session reference attributes (no endpoint_url):
class boto3.session.Session(aws_access_key_id=None, aws_secret_access_key=None, aws_session_token=None, region_name=None, botocore_session=None, profile_name=None)Boto3 Session does not support endpoint_url argument.
This argument must be added when creating s3 boto resource.
What could be done is something like this in kev.backends.s3.db.S3DB class constructor:
def __init__(self,**kwargs):
#
session_kwargs = {k: v for k, v in kwargs.items() if k in
self.session_kwargs}
endpoint_url = None
if 'endpoint_url' in session_kwargs.keys():
endpoint_url = session_kwargs['endpoint_url']
del(session_kwargs['endpoint_url'])
if len(session_kwargs.keys()) > 0:
boto3.Session(**session_kwargs)
self._db = boto3.resource('s3', endpoint_url=endpoint_url)
self.bucket = kwargs['bucket']
self._indexer = self._db.Bucket(self.bucket)boto3.resource reference attributes (existing endpoint_url kwarg):
resource(service_name, region_name=None, api_version=None, use_ssl=True, verify=None, endpoint_url=None, aws_access_key_id=None, aws_secret_access_key=None, aws_session_token=None, config=None)[source]endpoint_url default value is None, then this code should always be ok.
What do you think about this fix / workaround ?
Edit: We should add support to region_name also...