boto3
boto3 copied to clipboard
boto
SSLError: SSL validation failed ... EOF occurred in violation of protocol
Describe the bug
Python: 3.10.2 boto3/botocore: 1.24.17 OS: Alpine Linux 3.15
When attempting to upload a file to s3 I receive the following exception:
Traceback (most recent call last):
File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 396, in _make_request
conn.request_chunked(method, url, **httplib_request_kw)
File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/connection.py", line 275, in request_chunked
self.send(to_send)
File "/opt/app/.venv/lib/python3.10/site-packages/botocore/awsrequest.py", line 218, in send
return super().send(str)
File "/usr/local/lib/python3.10/http/client.py", line 998, in send
self.sock.sendall(data)
File "/usr/local/lib/python3.10/ssl.py", line 1236, in sendall
v = self.send(byte_view[count:])
File "/usr/local/lib/python3.10/ssl.py", line 1205, in send
return self._sslobj.write(data)
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2384)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/app/.venv/lib/python3.10/site-packages/botocore/httpsession.py", line 448, in send
urllib_response = conn.urlopen(
File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 785, in urlopen
retries = retries.increment(
File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/util/retry.py", line 525, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/packages/six.py", line 769, in reraise
raise value.with_traceback(tb)
File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 396, in _make_request
conn.request_chunked(method, url, **httplib_request_kw)
File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/connection.py", line 275, in request_chunked
self.send(to_send)
File "/opt/app/.venv/lib/python3.10/site-packages/botocore/awsrequest.py", line 218, in send
return super().send(str)
File "/usr/local/lib/python3.10/http/client.py", line 998, in send
self.sock.sendall(data)
File "/usr/local/lib/python3.10/ssl.py", line 1236, in sendall
v = self.send(byte_view[count:])
File "/usr/local/lib/python3.10/ssl.py", line 1205, in send
return self._sslobj.write(data)
urllib3.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:2384)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/app/.venv/lib/python3.10/site-packages/dramatiq/worker.py", line 485, in process_message
res = actor(*message.args, **message.kwargs)
File "/opt/app/.venv/lib/python3.10/site-packages/dramatiq/actor.py", line 145, in __call__
return self.fn(*args, **kwargs)
File "/opt/app/src/bona/tasks/actors/uploads.py", line 54, in move_upload_to_s3
s3.upload_file(
File "/opt/app/.venv/lib/python3.10/site-packages/boto3/s3/inject.py", line 143, in upload_file
return transfer.upload_file(
File "/opt/app/.venv/lib/python3.10/site-packages/boto3/s3/transfer.py", line 288, in upload_file
future.result()
File "/opt/app/.venv/lib/python3.10/site-packages/s3transfer/futures.py", line 103, in result
return self._coordinator.result()
File "/opt/app/.venv/lib/python3.10/site-packages/s3transfer/futures.py", line 266, in result
raise self._exception
File "/opt/app/.venv/lib/python3.10/site-packages/s3transfer/tasks.py", line 139, in __call__
return self._execute_main(kwargs)
File "/opt/app/.venv/lib/python3.10/site-packages/s3transfer/tasks.py", line 162, in _execute_main
return_value = self._main(**kwargs)
File "/opt/app/.venv/lib/python3.10/site-packages/s3transfer/upload.py", line 758, in _main
client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
File "/opt/app/.venv/lib/python3.10/site-packages/botocore/client.py", line 508, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/opt/app/.venv/lib/python3.10/site-packages/botocore/client.py", line 898, in _make_api_call
http, parsed_response = self._make_request(
File "/opt/app/.venv/lib/python3.10/site-packages/botocore/client.py", line 921, in _make_request
return self._endpoint.make_request(operation_model, request_dict)
File "/opt/app/.venv/lib/python3.10/site-packages/botocore/endpoint.py", line 119, in make_request
return self._send_request(request_dict, operation_model)
File "/opt/app/.venv/lib/python3.10/site-packages/botocore/endpoint.py", line 202, in _send_request
while self._needs_retry(
File "/opt/app/.venv/lib/python3.10/site-packages/botocore/endpoint.py", line 354, in _needs_retry
responses = self._event_emitter.emit(
File "/opt/app/.venv/lib/python3.10/site-packages/botocore/hooks.py", line 412, in emit
return self._emitter.emit(aliased_event_name, **kwargs)
File "/opt/app/.venv/lib/python3.10/site-packages/botocore/hooks.py", line 256, in emit
return self._emit(event_name, kwargs)
File "/opt/app/.venv/lib/python3.10/site-packages/botocore/hooks.py", line 239, in _emit
response = handler(**kwargs)
File "/opt/app/.venv/lib/python3.10/site-packages/botocore/retryhandler.py", line 207, in __call__
if self._checker(**checker_kwargs):
File "/opt/app/.venv/lib/python3.10/site-packages/botocore/retryhandler.py", line 284, in __call__
should_retry = self._should_retry(
File "/opt/app/.venv/lib/python3.10/site-packages/botocore/retryhandler.py", line 320, in _should_retry
return self._checker(attempt_number, response, caught_exception)
File "/opt/app/.venv/lib/python3.10/site-packages/botocore/retryhandler.py", line 363, in __call__
checker_response = checker(
File "/opt/app/.venv/lib/python3.10/site-packages/botocore/retryhandler.py", line 247, in __call__
return self._check_caught_exception(
File "/opt/app/.venv/lib/python3.10/site-packages/botocore/retryhandler.py", line 416, in _check_caught_exception
raise caught_exception
File "/opt/app/.venv/lib/python3.10/site-packages/botocore/endpoint.py", line 281, in _do_get_response
http_response = self._send(request)
File "/opt/app/.venv/lib/python3.10/site-packages/botocore/endpoint.py", line 377, in _send
return self.http_session.send(request)
File "/opt/app/.venv/lib/python3.10/site-packages/botocore/httpsession.py", line 475, in send
raise SSLError(endpoint_url=request.url, error=e)
botocore.exceptions.SSLError: SSL validation failed for https://bucket.s3.us-east-2.amazonaws.com/key/parts.ext EOF occurred in violation of protocol (_ssl.c:2384)
Expected Behavior
File should be uploaded to s3
Current Behavior
getting the EOF exception
Reproduction Steps
This error is being generated by the following:
s3: S3Client = boto3.client('s3')
s3.upload_file(
'local/path.ext',
'bucket-name',
'new/path.ext',
ExtraArgs={'ChecksumAlgorithm': 'SHA256'},
)
I've been able to reproduce a similar error against s3 and other ssl sites using the following.
from botocore.httpsession import create_urllib3_context
import urllib3
http = urllib3.PoolManager(ssl_context=create_urllib3_context(), retries=False)
http.urlopen('GET', 'https://www.howsmyssl.com/a/check')
Traceback (most recent call last):
File "<input>", line 1, in <module>
http.urlopen('GET', 'https://www.howsmyssl.com/a/check')
File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/poolmanager.py", line 376, in urlopen
response = conn.urlopen(method, u.request_uri, **kw)
File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 785, in urlopen
retries = retries.increment(
File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/util/retry.py", line 525, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/packages/six.py", line 769, in reraise
raise value.with_traceback(tb)
File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 386, in _make_request
self._validate_conn(conn)
File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1040, in _validate_conn
conn.connect()
File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/connection.py", line 414, in connect
self.sock = ssl_wrap_socket(
File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
ssl_sock = _ssl_wrap_socket_impl(
File "/opt/app/.venv/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/local/lib/python3.10/ssl.py", line 512, in wrap_socket
return self.sslsocket_class._create(
File "/usr/local/lib/python3.10/ssl.py", line 1070, in _create
self.do_handshake()
File "/usr/local/lib/python3.10/ssl.py", line 1341, in do_handshake
self._sslobj.do_handshake()
urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)
Possible Solution
No response
Additional Information/Context
No response
SDK version used
1.24.17
Environment details (OS name and version, etc.)
Alpine Linux 3.15, Running in EKS
I'm getting the same thing using windows 10 with python uploading a file. I'm using boto3 version 1.24.27. I recently upgraded my packages and this started to happen then. I don't use a proxy. I'm using python 3.10.5 and botocore 1.27.27.
Hi @Kaelten & @jschwar313 thanks for reaching out. That error has been reported several times across our repositories. (Both the AWS CLI and boto3 use botocore). Have you tried looking through those issues for solutions? If this has just started happening recently then we should rule out if any recent changes are causing this. Which version of urllib3 are you both using?
These are my python packages. I hope I'm not hijacking this thread. I apologize if I am.
C:\Windows\system32>pip list Package Version
altgraph 0.17.2 boto3 1.24.27 botocore 1.27.27 certifi 2022.6.15 charset-normalizer 2.0.12 cx-Freeze 6.11.1 cx-Logging 3.0 Cython 0.29.30 future 0.18.2 idna 3.3 importlib-metadata 4.12.0 jmespath 1.0.1 lief 0.12.1 packaging 21.3 pefile 2022.5.30 pip 22.2 psutil 5.9.1 pyinstaller 5.2 pyinstaller-hooks-contrib 2022.8 pyparsing 3.0.9 python-dateutil 2.8.2 pywin32-ctypes 0.2.0 requests 2.28.1 s3transfer 0.6.0 setuptools 60.10.0 six 1.16.0 urllib3 1.26.10 wheel 0.37.1 zipp 3.8.0
Do you think upgrading will help?
C:\Windows\system32>pip3 list --outdated Package Version Latest Type
boto3 1.24.27 1.24.38 wheel botocore 1.27.27 1.27.38 wheel charset-normalizer 2.0.12 2.1.0 wheel Cython 0.29.30 0.29.31 wheel pip 22.2 22.2.1 wheel setuptools 60.10.0 63.2.0 wheel urllib3 1.26.10 1.26.11 wheel zipp 3.8.0 3.8.1 wheel
I've upgraded a number of packages and that didn't help. I'm still getting that error now. Here's what I have now:
C:\Windows\system32>pip3 list --outdated Package Version Latest Type
charset-normalizer 2.0.12 2.1.0 wheel Cython 0.29.30 0.29.31 wheel setuptools 60.10.0 63.2.0 wheel zipp 3.8.0 3.8.1 wheel
C:\Windows\system32>pip3 list Package Version
altgraph 0.17.2 boto3 1.24.39 botocore 1.27.39 certifi 2022.6.15 cffi 1.15.1 charset-normalizer 2.0.12 cryptography 37.0.4 cx-Freeze 6.11.1 cx-Logging 3.0 Cython 0.29.30 future 0.18.2 idna 3.3 importlib-metadata 4.12.0 jmespath 1.0.1 lief 0.12.1 packaging 21.3 pefile 2022.5.30 pip 22.2.1 psutil 5.9.1 pycparser 2.21 pyinstaller 5.2 pyinstaller-hooks-contrib 2022.8 pyOpenSSL 22.0.0 pyparsing 3.0.9 python-dateutil 2.8.2 pywin32-ctypes 0.2.0 requests 2.28.1 s3transfer 0.6.0 setuptools 60.10.0 six 1.16.0 urllib3 1.26.11 wheel 0.37.1 zipp 3.8.0
Hi @Kaelten & @jschwar313 thanks for reaching out. That error has been reported several times across our repositories. (Both the AWS CLI and boto3 use botocore). Have you tried looking through those issues for solutions? If this has just started happening recently then we should rule out if any recent changes are causing this. Which version of urllib3 are you both using?
I missed the reply notifications sorry for the lag responding.
Thanks to some help in a discord server I was able to figure out the underlying cause, and it has nothing to do with my SSL configuration.
First, I learned that this was related to multipart uploads. By using this transfer config I was able to get bypass the SSLError.
Config = TransferConfig(multipart_threshold=1)
HOWEVER, what I found when I did that was this had nothing to do with SSL at all. My s3 buckets are using kms key encryption and I was missing kms:Decrypt for the respective key on service role. When bypassing the SSLError I received a perfectly cogent error message telling me exactly what was missing and how to fix it.
After adding this permission, I was able to remove the above TransferConfig and the uploads remain error free. All in all I spent almost 12 hours debugging only to find it was a 3 second permissions fix because boto was somehow surfacing a SSLError for a permissions issue.
I hope this helps you guys get to the bottom of at least once source of these errors.
I checked the buckets I'm getting the SSLError on when uploading and the properties for my buckets has "default encryption" disabled. Does that mean this can't be the problem for my case?
@Kaelten By the way, since you are obviously doing multipart uploads, you should check out this article. You may be charged for parts that failed and are still in your s3 bucket. This is how you can get rid of those parts. I was being overcharged this way. Personally, I think this is a class action suit waiting to happen, but I won't do it. https://aws.amazon.com/blogs/aws/s3-lifecycle-management-update-support-for-multipart-uploads-and-delete-markers/
I tried creating a policy and attaching it to my user, but I'm still getting that error. I've attached the policy that I created. It doesn't fail the upload, but it does fail the part.
{ "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": [ "kms:Decrypt", "kms:Encrypt" ], "Resource": "arn:aws:kms:::key/" } ] }
I corrected the error in my policy and tried it again. The error still shows up. Here's the new policy I'm using.
{ "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": [ "kms:Decrypt", "kms:Encrypt" ], "Resource": "*" } ] }
The error stopped happening. I don't think it was a python package upgrade that did it. I'm assuming it was a windows update that helped.
Nope. it's still happening. It just didn't happen that one time I uploaded a large file. It's happening again. I will upgrade my packages again and see if that fixes this. I doubt it. Here's what I have now:
C:\Windows\system32>python --version Python 3.10.5
C:\Windows\system32>pip3 list Package Version
altgraph 0.17.2 boto3 1.24.39 botocore 1.27.39 certifi 2022.6.15 cffi 1.15.1 charset-normalizer 2.0.12 cryptography 37.0.4 cx-Freeze 6.11.1 cx-Logging 3.0 Cython 0.29.30 future 0.18.2 idna 3.3 importlib-metadata 4.12.0 jmespath 1.0.1 lief 0.12.1 packaging 21.3 pefile 2022.5.30 pip 22.2.2 psutil 5.9.1 pycparser 2.21 pyinstaller 5.2 pyinstaller-hooks-contrib 2022.8 pyOpenSSL 22.0.0 pyparsing 3.0.9 python-dateutil 2.8.2 pywin32-ctypes 0.2.0 requests 2.28.1 s3transfer 0.6.0 setuptools 60.10.0 six 1.16.0 urllib3 1.26.11 wheel 0.37.1 zipp 3.8.0
C:\Windows\system32>pip3 list --outdated Package Version Latest Type
boto3 1.24.39 1.24.47 wheel botocore 1.27.39 1.27.47 wheel charset-normalizer 2.0.12 2.1.0 wheel Cython 0.29.30 0.29.32 wheel pyinstaller 5.2 5.3 wheel setuptools 60.10.0 63.4.2 wheel zipp 3.8.0 3.8.1 wheel
C:\Windows\system32>
It's not a fatal error that stops my process. Is that true for everyone? If not, I have the following lines below my region in my config file for AWS.
max_attempts = 100 retry_mode = legacy
I had the same problem SSL validation failed for https://karbon-text.s3.ap-south-1.amazonaws.com/upload.pdf EOF occurred in violation of protocol (_ssl.c:877) Oddly, I didn't have any problems on my own computer, but it happened when I used the upload file on the server
I had the same problem SSL validation failed for https://karbon-text.s3.ap-south-1.amazonaws.com/upload.pdf EOF occurred in violation of protocol (_ssl.c:877) Oddly, I didn't have any problems on my own computer, but it happened when I used the upload file on the server
I do not have any problem with macos and Windows, but server with centos will report errors when uploading files
I had the same problem SSL validation failed for https://karbon-text.s3.ap-south-1.amazonaws.com/upload.pdf EOF occurred in violation of protocol (_ssl.c:877) Oddly, I didn't have any problems on my own computer, but it happened when I used the upload file on the server
I do not have any problem with macos and Windows, but server with centos will report errors when uploading files
2022-10-14 03:02:44,022 project.invoice.untils INFO <botocore.client.S3 object at 0x7f5538426908> before upload
2022-10-14 03:02:54,501 django.request ERROR Internal Server Error: /invoice/upload-file-to-aws
Traceback (most recent call last):
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 594, in urlopen
self._prepare_proxy(conn)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 805, in _prepare_proxy
conn.connect()
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/urllib3/connection.py", line 344, in connect
ssl_context=context)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/urllib3/util/ssl_.py", line 345, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib64/python3.6/ssl.py", line 365, in wrap_socket
_context=self, _session=session)
File "/usr/lib64/python3.6/ssl.py", line 773, in __init__
self.do_handshake()
File "/usr/lib64/python3.6/ssl.py", line 1033, in do_handshake
self._sslobj.do_handshake()
File "/usr/lib64/python3.6/ssl.py", line 645, in do_handshake
self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:877)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/botocore/httpsession.py", line 263, in send
chunked=self._chunked(request.headers),
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/urllib3/util/retry.py", line 344, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/urllib3/packages/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 594, in urlopen
self._prepare_proxy(conn)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 805, in _prepare_proxy
conn.connect()
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/urllib3/connection.py", line 344, in connect
ssl_context=context)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/urllib3/util/ssl_.py", line 345, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib64/python3.6/ssl.py", line 365, in wrap_socket
_context=self, _session=session)
File "/usr/lib64/python3.6/ssl.py", line 773, in __init__
self.do_handshake()
File "/usr/lib64/python3.6/ssl.py", line 1033, in do_handshake
self._sslobj.do_handshake()
File "/usr/lib64/python3.6/ssl.py", line 645, in do_handshake
self._sslobj.do_handshake()
urllib3.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:877)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/django/core/handlers/base.py", line 183, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/django/core/handlers/base.py", line 339, in process_exception_by_middleware
response = middleware_method(request, exception)
File "/home/kbuser/f4/karbon_app/karbon_backend/project/common/middleware.py", line 29, in process_exception
raise exception
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/django/core/handlers/base.py", line 181, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
return view_func(*args, **kwargs)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/django/views/generic/base.py", line 70, in view
return self.dispatch(request, *args, **kwargs)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/rest_framework/views.py", line 505, in dispatch
response = self.handle_exception(exc)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/rest_framework/views.py", line 465, in handle_exception
self.raise_uncaught_exception(exc)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/rest_framework/views.py", line 476, in raise_uncaught_exception
raise exc
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/rest_framework/views.py", line 502, in dispatch
response = handler(request, *args, **kwargs)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/rest_framework/generics.py", line 190, in post
return self.create(request, *args, **kwargs)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/rest_framework/mixins.py", line 19, in create
self.perform_create(serializer)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/rest_framework/mixins.py", line 24, in perform_create
serializer.save()
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/rest_framework/serializers.py", line 212, in save
self.instance = self.create(validated_data)
File "/home/kbuser/f4/karbon_app/karbon_backend/project/invoice/serializers/invoice.py", line 451, in create
res = upload_file_to_aws(instance.extract_file.path)
File "/home/kbuser/f4/karbon_app/karbon_backend/project/invoice/untils.py", line 40, in upload_file_to_aws
s3_client.upload_file(file_name, bucket, object_name)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/boto3/s3/inject.py", line 131, in upload_file
extra_args=ExtraArgs, callback=Callback)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/boto3/s3/transfer.py", line 279, in upload_file
future.result()
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/s3transfer/futures.py", line 106, in result
return self._coordinator.result()
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/s3transfer/futures.py", line 265, in result
raise self._exception
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/s3transfer/tasks.py", line 126, in __call__
return self._execute_main(kwargs)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/s3transfer/tasks.py", line 150, in _execute_main
return_value = self._main(**kwargs)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/s3transfer/upload.py", line 692, in _main
client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/botocore/client.py", line 316, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/botocore/client.py", line 613, in _make_api_call
operation_model, request_dict, request_context)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/botocore/client.py", line 632, in _make_request
return self._endpoint.make_request(operation_model, request_dict)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/botocore/endpoint.py", line 102, in make_request
return self._send_request(request_dict, operation_model)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/botocore/endpoint.py", line 137, in _send_request
success_response, exception):
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/botocore/endpoint.py", line 256, in _needs_retry
caught_exception=caught_exception, request_dict=request_dict)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/botocore/hooks.py", line 356, in emit
return self._emitter.emit(aliased_event_name, **kwargs)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/botocore/hooks.py", line 228, in emit
return self._emit(event_name, kwargs)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/botocore/hooks.py", line 211, in _emit
response = handler(**kwargs)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 183, in __call__
if self._checker(attempts, response, caught_exception):
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 251, in __call__
caught_exception)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 277, in _should_retry
return self._checker(attempt_number, response, caught_exception)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 317, in __call__
caught_exception)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 223, in __call__
attempt_number, caught_exception)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
raise caught_exception
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/botocore/endpoint.py", line 200, in _do_get_response
http_response = self._send(request)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/botocore/endpoint.py", line 269, in _send
return self.http_session.send(request)
File "/home/kbuser/f4/karbon_app/python36-venv/lib64/python3.6/site-packages/botocore/httpsession.py", line 281, in send
raise SSLError(endpoint_url=request.url, error=e)
botocore.exceptions.SSLError: SSL validation failed for https://karbon-text.s3.ap-south-1.amazonaws.com/Snipaste_2022-08-30_15-01-07.png EOF occurred in violation of protocol (_ssl.c:877)
I upgraded my python and packages and it didn't happen the last two uploads that I've done, which take quite a while. (They are on my home PC.) Not sure if that means this is fixed or it might happen next time. My next upload is going to happen Thursday to Friday.
I upgraded my python and packages and it didn't happen the last two uploads that I've done, which take quite a while. (They are on my home PC.) Not sure if that means this is fixed or it might happen next time. My next upload is going to happen Thursday to Friday.
Can you tell me your previous python and packages versions and the upgraded versions，thank you！！！
I listed my python version and packages previously in this post. I'm now using python version 3.10.8
My packages are:
Package Version
altgraph 0.17.3 attrs 22.1.0 boto3 1.24.90 botocore 1.27.90 certifi 2022.9.24 cffi 1.15.1 charset-normalizer 2.1.1 colorama 0.4.5 cryptography 38.0.1 cx-Freeze 6.12.0 cx-Logging 3.0 Cython 0.29.32 future 0.18.2 idna 3.4 importlib-metadata 5.0.0 iniconfig 1.1.1 jmespath 1.0.1 lief 0.12.2 packaging 21.3 pefile 2022.5.30 pip 22.2.2 pluggy 1.0.0 psutil 5.9.2 py 1.11.0 pycparser 2.21 pyinstaller 5.5 pyinstaller-hooks-contrib 2022.10 pyOpenSSL 22.1.0 pyparsing 3.0.9 pytest 7.1.3 python-dateutil 2.8.2 pywin32-ctypes 0.2.0 requests 2.28.1 s3transfer 0.6.0 setuptools 65.5.0 six 1.16.0 tomli 2.0.1 urllib3 1.26.12 wheel 0.37.1 zipp 3.9.0
I listed my python version and packages previously in this post. I'm now using python version 3.10.8
My packages are:
Package Version
altgraph 0.17.3 attrs 22.1.0 boto3 1.24.90 botocore 1.27.90 certifi 2022.9.24 cffi 1.15.1 charset-normalizer 2.1.1 colorama 0.4.5 cryptography 38.0.1 cx-Freeze 6.12.0 cx-Logging 3.0 Cython 0.29.32 future 0.18.2 idna 3.4 importlib-metadata 5.0.0 iniconfig 1.1.1 jmespath 1.0.1 lief 0.12.2 packaging 21.3 pefile 2022.5.30 pip 22.2.2 pluggy 1.0.0 psutil 5.9.2 py 1.11.0 pycparser 2.21 pyinstaller 5.5 pyinstaller-hooks-contrib 2022.10 pyOpenSSL 22.1.0 pyparsing 3.0.9 pytest 7.1.3 python-dateutil 2.8.2 pywin32-ctypes 0.2.0 requests 2.28.1 s3transfer 0.6.0 setuptools 65.5.0 six 1.16.0 tomli 2.0.1 urllib3 1.26.12 wheel 0.37.1 zipp 3.9.0
Thank you, and I found that the company server turned off the agent can be, I do not know whether the agent is the reason, but it can be used normally. However, I am using the proxy locally without any problems, so maybe turning the proxy off is also an option, although it is not normal
Checking in again - SSL validation errors are often attributed to certificate issues, for example see this section in the AWS CLI troubleshooting guide: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-troubleshooting.html#tshoot-certificate-verify-failed.
Although in this earlier comment the user who opened this issue described how the error was caused by kms:Decrypt not being attached to a service role. Can anyone else confirm that was causing this issue?
No. I added that permission and it didn't do any good. I recently took it off.
One thing I've notice when going through that AWS document is that aws --version lists a different version of python than I have installed. I'll uninstall the aws cli and reinistall it. I have python 3.11.0 installed. The aws --version says this:
aws-cli/2.4.6 Python/3.8.8 Windows/10 exe/AMD64 prompt/off
now it says this:
aws-cli/2.9.3 Python/3.9.11 Windows/10 exe/AMD64 prompt/off
I'll see if updating the aws-cli helps at all. I'll start an upload today.
Actually, I figured out a way to search my log files for "SSLEOFError" and this hasn't happened since 11/17/2022. I guess maybe it is fixed. Sorry about that.
Thanks for following up and confirming, I'll set this issue to auto-close unless there's anything else to discuss.
For me, upgrading from boto3 1.26.142 to 1.28.63 solved the issue.
I was trying to upload a csv to s3.
