aws-cloudformation
cfn-init crashes when run with python 3.10 on ubuntu 22.04
Resource Name
No response
Details
Running cfn-init from the https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-py3-latest.tar.gz package (as of 25/5/22) on an EC2 instance running ubuntu 22.04 fails with the following error:
/usr/local/bin/cfn-init -v --stack base-016smubuntu2204-EC2-6SV8YO1N4WBK --region eu-west-1 --resource EC2Instance
/usr/local/bin/cfn-init:115: SyntaxWarning: "is" with a literal. Did you mean "=="?
if type(e).__name__ is "ValueError" or type(e).__name__ is "JSONDecodeError":
/usr/local/bin/cfn-init:115: SyntaxWarning: "is" with a literal. Did you mean "=="?
if type(e).__name__ is "ValueError" or type(e).__name__ is "JSONDecodeError":
/usr/local/bin/cfn-init:133: SyntaxWarning: "is" with a literal. Did you mean "=="?
if type(e).__name__ is "ValueError" or type(e).__name__ is "JSONDecodeError":
/usr/local/bin/cfn-init:133: SyntaxWarning: "is" with a literal. Did you mean "=="?
if type(e).__name__ is "ValueError" or type(e).__name__ is "JSONDecodeError":
Traceback (most recent call last):
File "/usr/local/bin/cfn-init", line 21, in <module>
from cfnbootstrap.cfn_client import CloudFormationClient
File "/usr/local/lib/python3.10/dist-packages/cfnbootstrap/cfn_client.py", line 24, in <module>
from cfnbootstrap import aws_client, util
File "/usr/local/lib/python3.10/dist-packages/cfnbootstrap/aws_client.py", line 23, in <module>
from cfnbootstrap import util, public_constants
File "/usr/local/lib/python3.10/dist-packages/cfnbootstrap/util.py", line 22, in <module>
from cfnbootstrap.packages.requests.exceptions import ConnectionError, HTTPError, Timeout, SSLError
File "/usr/local/lib/python3.10/dist-packages/cfnbootstrap/packages/requests/__init__.py", line 58, in <module>
from . import utils
File "/usr/local/lib/python3.10/dist-packages/cfnbootstrap/packages/requests/utils.py", line 30, in <module>
from .cookies import RequestsCookieJar, cookiejar_from_dict
File "/usr/local/lib/python3.10/dist-packages/cfnbootstrap/packages/requests/cookies.py", line 159, in <module>
class RequestsCookieJar(cookielib.CookieJar, collections.MutableMapping):
AttributeError: module 'collections' has no attribute 'MutableMapping'
The cause is that the vendored requests package is incompatible with python 3.10, which is shipped as the default interpreter on this version of ubuntu.
I found a similar report here: https://github.com/psf/requests/issues/6013
One solution is to vendor a more recent version of requests in the cfn-bootstrap package.
We're trying to update our EC2 estate to ubuntu22.04 but this is a blocker until fixed.
ok, so why that's happening:
cfnbootstrap contains a copy of requests package, and that's quite an old version of requests
current version of requests supports python3.10
ok, so why that's happening:
cfnbootstrapcontains a copy ofrequestspackage, and that's quite an old version of requestscurrent version of requests supports python3.10
@arogozhnikov Did you try upgrading requests and see if it works? I did, but to no avail. Still the same error as described by OP.
I did swap in the current requests to replace what is vendored and then another package conflicted, but I don't remember the details.
I ended up going to Ubuntu 20.04 LTS in the end, since that ships with an older version of Python that still works. Hopefully we'll see a new version of the cloudformation helpers soon that is compatible with the current version of Python and packages etc. Shame it isn't open sourced and sitting in a repo where we can fix it more easily ourselves.
@smeyfroi Yeah I did the same, I mean going back to 20.04 on my project. Perhaps we can try installing an older python. But is that the reason why 20.04 and 18.04 don't experience this issue, that they have older pythons?
Yes it's Python 3.10 that's incompatible with the very old packages that the cfn tools ship with. 20.04 is fine because old version of Python.
We recently moved onto a recent release of 22.04 without incident. So it might "just work" if you try again; if not ping here and I'll dig into our cfn-init stuff for you to enjoy some copy-paste goodness.
We recently moved onto a recent release of 22.04 without incident. So it might "just work" if you try again; if not ping here and I'll dig into our cfn-init stuff for you to enjoy some copy-paste goodness.
Good to hear man. I'll do some testings on my own. Thanks for your help.