copr icon indicating copy to clipboard operation
copr copied to clipboard

cloud-init problem with F41 in EC2 - boot is delayed by ~2 minutes

Open praiskup opened this issue 1 year ago • 3 comments

[   32.258641] cloud-init[859]: 2024-11-28 00:59:08,009 - url_helper.py[WARNING]: Exception(s) [UrlError("HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /latest/api/token (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6eb44ac2f0>: Failed to establish a new connection: [Errno 101] Network is unreachable'))"), UrlError("HTTPConnectionPool(host='fd00:ec2::254', port=80): Max retries exceeded with url: /latest/api/token (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6eb44a96d0>: Failed to establish a new connection: [Errno 101] Network is unreachable'))")] during request to http://[fd00:ec2::254]/latest/api/token, raising last exception
[   32.566342] cloud-init[859]: 2024-11-28 00:59:08,009 - url_helper.py[WARNING]: Calling 'http://[fd00:ec2::254]/latest/api/token' failed [0/120s]: request error [HTTPConnectionPool(host='fd00:ec2::254', port=80): Max retries exceeded with url: /latest/api/token (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6eb44a96d0>: Failed to establish a new connection: [Errno 101] Network is unreachable'))]
[   83.261948] cloud-init[859]: 2024-11-28 00:59:59,012 - url_helper.py[WARNING]: Timed out waiting for addresses: http://169.254.169.254/latest/api/token http://[fd00:ec2::254]/latest/api/token, exception(s) raised while waiting: HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /latest/api/token (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6eb44aa710>: Failed to establish a new connection: [Errno 101] Network is unreachable'))
[   83.469744] cloud-init[859]: 2024-11-28 00:59:59,012 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/latest/api/token' failed [51/120s]: unexpected error ['NoneType' object has no attribute 'contents']
[  134.264155] cloud-init[859]: 2024-11-28 01:00:50,014 - url_helper.py[WARNING]: Timed out waiting for addresses: http://169.254.169.254/latest/api/token http://[fd00:ec2::254]/latest/api/token, exception(s) raised while waiting: HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /latest/api/token (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6eb4477950>: Failed to establish a new connection: [Errno 101] Network is unreachable'))
[  134.468954] cloud-init[859]: 2024-11-28 01:00:50,015 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/latest/api/token' failed [102/120s]: unexpected error ['NoneType' object has no attribute 'contents']
[  152.102478] cloud-init[859]: 2024-11-28 01:01:07,853 - url_helper.py[WARNING]: Timed out waiting for addresses: http://169.254.169.254/latest/api/token http://[fd00:ec2::254]/latest/api/token, exception(s) raised while waiting: HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /latest/api/token (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6eb44d46b0>: Failed to establish a new connection: [Errno 101] Network is unreachable'))
[  152.312364] cloud-init[859]: 2024-11-28 01:01:07,853 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/latest/api/token' failed [120/120s]: unexpected error ['NoneType' object has no attribute 'contents']
[  OK  ] Finished cloud-init-local.service …ud-init: Local Stage (pre-network).
[  152.399305] cloud-init[859]: 2024-11-28 01:01:07,853 - url_helper.py[ERROR]: Timed out, no response from urls: ['http://169.254.169.254/latest/api/token', 'http://[fd00:ec2::254]/latest/api/token']

praiskup avatar Nov 28 '24 01:11 praiskup

[   35.649693] cloud-init[894]: 2024-11-29 17:15:29,844 - url_helper.py[WARNING]: Exception(s) [UrlError("HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /latest/api/token (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f12b6b642f0>: Failed to establish a new connection: [Errno 101] Network is unreachable'))"), UrlError("HTTPConnectionPool(host='fd00:ec2::254', port=80): Max retries exceeded with url: /latest/api/token (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f12b6b616d0>: Failed to establish a new connection: [Errno 101] Network is unreachable'))")] during request to http://[fd00:ec2::254]/latest/api/token, raising last exception
[   35.956962] cloud-init[894]: 2024-11-29 17:15:29,844 - url_helper.py[WARNING]: Calling 'http://[fd00:ec2::254]/latest/api/token' failed [0/120s]: request error [HTTPConnectionPool(host='fd00:ec2::254', port=80): Max retries exceeded with url: /latest/api/token (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f12b6b616d0>: Failed to establish a new connection: [Errno 101] Network is unreachable'))]
[   36.803248] cloud-init[894]: 2024-11-29 17:15:30,998 - url_helper.py[WARNING]: Exception(s) [UrlError("HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /latest/api/token (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f12b6b62710>: Failed to establish a new connection: [Errno 101] Network is unreachable'))"), UrlError("HTTPConnectionPool(host='fd00:ec2::254', port=80): Max retries exceeded with url: /latest/api/token (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f12b6b2efd0>: Failed to establish a new connection: [Errno 101] Network is unreachable'))")] during request to http://[fd00:ec2::254]/latest/api/token, raising last exception
[   37.110891] cloud-init[894]: 2024-11-29 17:15:30,998 - url_helper.py[WARNING]: Calling 'http://[fd00:ec2::254]/latest/api/token' failed [1/120s]: request error [HTTPConnectionPool(host='fd00:ec2::254', port=80): Max retries exceeded with url: /latest/api/token (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f12b6b2efd0>: Failed to establish a new connection: [Errno 101] Network is unreachable'))]
[   87.805883] cloud-init[894]: 2024-11-29 17:16:22,000 - url_helper.py[WARNING]: Timed out waiting for addresses: http://169.254.169.254/latest/api/token http://[fd00:ec2::254]/latest/api/token, exception(s) raised while waiting: HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /latest/api/token (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f12b6b2f950>: Failed to establish a new connection: [Errno 101] Network is unreachable'))
[   88.012927] cloud-init[894]: 2024-11-29 17:16:22,000 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/latest/api/token' failed [52/120s]: unexpected error ['NoneType' object has no attribute 'contents']
[  138.809513] cloud-init[894]: 2024-11-29 17:17:13,003 - url_helper.py[WARNING]: Timed out waiting for addresses: http://169.254.169.254/latest/api/token http://[fd00:ec2::254]/latest/api/token, exception(s) raised while waiting: HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /latest/api/token (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f12b6b8c6b0>: Failed to establish a new connection: [Errno 101] Network is unreachable'))
[  139.013615] cloud-init[894]: 2024-11-29 17:17:13,004 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/latest/api/token' failed [103/120s]: unexpected error ['NoneType' object has no attribute 'contents']
[  155.494943] cloud-init[894]: 2024-11-29 17:17:29,689 - url_helper.py[WARNING]: Timed out waiting for addresses: http://169.254.169.254/latest/api/token http://[fd00:ec2::254]/latest/api/token, exception(s) raised while waiting: HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /latest/api/token (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f12b6b6f550>: Failed to establish a new connection: [Errno 101] Network is unreachable'))
[  155.701594] cloud-init[894]: 2024-11-29 17:17:29,689 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/latest/api/token' failed [120/120s]: unexpected error ['NoneType' object has no attribute 'contents']
[  155.791469] cloud-init[894]: 2024-11-29 17:17:29,690 - url_helper.py[ERROR]: Timed out, no response from urls: ['http://169.254.169.254/latest/api/token', 'http://[fd00:ec2::254]/latest/api/token']
[  OK  ] Finished cloud-init-local.service …ud-init: Local Stage (pre-network).

When I try to curl -X PUT http://169.254.169.254/latest/api/token -H "X-aws-ec2-metadata-token-ttl-seconds: 21600", the request works.

praiskup avatar Nov 29 '24 17:11 praiskup

This is Starting cloud-init-local.service …-init: Local Stage (pre-network)... contacting network doesn't feel right.

praiskup avatar Nov 29 '24 18:11 praiskup

Reported against Fedora's cloud-init: https://bugzilla.redhat.com/show_bug.cgi?id=2329833

praiskup avatar Dec 02 '24 10:12 praiskup