amazon-ssm-agent icon indicating copy to clipboard operation
amazon-ssm-agent copied to clipboard

Updating from Ubuntu 20.04.1 to 22.04.19 broke SSM link

Open ophilli opened this issue 1 year ago • 4 comments

Problem:

After updating from Ubuntu 20.04.1 to 22.04.19 my host began failing to connect to AWS Systems Manager. Notably with the following error: https://github.com/aws/amazon-ssm-agent/blob/0117b6eb82282b50acc4299f8066efeffc509a7c/agent/managedInstances/fingerprint/fingerprint.go#L301

I also had the following warnings:

WARN [OnPremCredsProvider] The 'IP Address' value (REDACTED) has changed from the registered machine configuration value (REDACTED).
WARN [OnPremCredsProvider] The 'processor-hash' value (REDACTED) has changed from the registered machine configuration value (REDACTED).
WARN [OnPremCredsProvider] The 'ipaddress-info' value (REDACTED) has changed from the registered machine configuration value (REDACTED).
WARN [OnPremCredsProvider] The 'disk-info' value (REDACTED) has changed from the registered machine configuration value (REDACTED).
WARN [OnPremCredsProvider] The 'memory-hash' value (REDACTED) has changed from the registered machine configuration value (REDACTED).
WARN [OnPremCredsProvider] The 'bios-hash' value (REDACTED) has changed from the registered machine configuration value (REDACTED).
WARN [OnPremCredsProvider] The 'system-hash' value (REDACTED) has changed from the registered machine configuration value (REDACTED).
INFO [OnPremCredsProvider] Calculated hardware difference, regenerating fingerprint...
ERROR [CredentialRefresher] Retrieve credentials produced error: MachineFingerprintDoesNotMatch: Fingerprint does not match REDACTED

ophilli avatar May 07 '24 04:05 ophilli

https://www.reddit.com/r/aws/comments/scobtu/aws_ssm_machine_fingerprint_changed_after_os/ suggests that I may need to aws ssm create-activation again.

ophilli avatar May 07 '24 04:05 ophilli

https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent-technical-details.html#fingerprint-validation seems relevant

ophilli avatar May 07 '24 04:05 ophilli

I ended up running create-activation again and now I'm up and running but I have a new managed id unfortunately.

It would be nice if the ssm agent were more resilient to distro upgrades, or had better documentation about how to manage this process.

ophilli avatar May 07 '24 07:05 ophilli

Having the same issue as well. Patched one of my systems using Patch Manager, and the new fingerprint caused the machine to stop reporting in with SSM. Hopefully Amazon can get a fix for this, because I definitely have reservations about using Patch Manager now.

drzewiec avatar Oct 28 '24 18:10 drzewiec