EC2MetadataUtils used to determine if running on cloud environment does not work on ECS Fargate

[trouptelia]

Type: Bug

Describe the bug Version of Spring Cloud: org.springframework.cloud:spring-cloud-dependencies:Hoxton.SR9 Which resolves to spring cloud aws version 2.2.5.RELEASE

When the application starts up the org.springframework.cloud.aws.context.support.env.AwsCloudEnvironmentCheckUtils attempts to determine if the code is running in a cloud environment by using com.amazonaws.util.EC2MetadataUtils. This does not appear to work if the code is deployed to ECS Fargate.

2020-11-28 00:34:36.424  WARN 1 --- [main] i.InstanceMetadataServiceResourceFetcher : Fail to retrieve token
--
com.amazonaws.SdkClientException: Failed to connect to service endpoint:
at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:100)
at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.getToken(InstanceMetadataServiceResourceFetcher.java:91)
at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.readResource(InstanceMetadataServiceResourceFetcher.java:69)
at com.amazonaws.internal.EC2ResourceFetcher.readResource(EC2ResourceFetcher.java:66)
at com.amazonaws.util.EC2MetadataUtils.getItems(EC2MetadataUtils.java:402)
at com.amazonaws.util.EC2MetadataUtils.getData(EC2MetadataUtils.java:371)
at org.springframework.cloud.aws.context.support.env.AwsCloudEnvironmentCheckUtils.isRunningOnCloudEnvironment(AwsCloudEnvironmentCheckUtils.java:38)
at org.springframework.cloud.aws.context.annotation.OnAwsCloudEnvironmentCondition.matches(OnAwsCloudEnvironmentCondition.java:38)
at org.springframework.context.annotation.ConditionEvaluator.shouldSkip(ConditionEvaluator.java:108)
at org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader$TrackedConditionEvaluator.shouldSkip(ConfigurationClassBeanDefinitionReader.java:477)
at org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader.loadBeanDefinitionsForConfigurationClass(ConfigurationClassBeanDefinitionReader.java:131)
at org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader.loadBeanDefinitions(ConfigurationClassBeanDefinitionReader.java:120)
at org.springframework.context.annotation.ConfigurationClassPostProcessor.processConfigBeanDefinitions(ConfigurationClassPostProcessor.java:331)
at org.springframework.context.annotation.ConfigurationClassPostProcessor.postProcessBeanDefinitionRegistry(ConfigurationClassPostProcessor.java:236)
at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanDefinitionRegistryPostProcessors(PostProcessorRegistrationDelegate.java:280)
at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:96)
at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:707)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:533)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:143)

I have already set the following in my application.yml:

cloud:
  aws:
    region:
      auto: true
      use-default-aws-region-chain: true
    credentials:
      use-default-aws-credentials-chain: true

As described in PR: https://github.com/spring-cloud/spring-cloud-aws/pull/559

But this seems to be yet another case where the class being used needs to be configurable based on the platform in AWS the code is deployed to EC2 vs ECS Fargate.

Sample Deploy any spring app using the spring cloud aws version 2.2.5.RELEASE to ECS Fargate and you should see this issue

[maciejwalkowiak]

Thanks for the report. Indeed ConditionalOnAwsCloudEnvironment does not work on Fargate. In your setup, exclude ContextInstanceDataAutoConfiguration as this is the only one that uses this conditional annotation and the warning message will disappear, also, your application will startup faster.

Starting from 2.3.0 ContextInstanceDataAutoConfiguration will be disabled by default.

[buckett]

Is the plan for this to improve org.springframework.cloud.aws.context.support.env.AwsCloudEnvironmentCheckUtils so that it understands the metadata endpoints documented in: https://docs.aws.amazon.com/AmazonECS/latest/userguide/task-metadata-endpoint-fargate.html

[buckett]

I had a look in the AWS v1 SDK (current SDK for Spring Cloud AWS) with https://github.com/aws/aws-sdk-java/search?q=169.254.170&type=code but there doesn't appear (only from 10 minutes looking) to be a way to get the ECS metadata, the only thing I found was a way to load the container credentials: https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-core/src/main/java/com/amazonaws/auth/ContainerCredentialsProvider.java

[kbroadbent]

Under the hood the AwsCloudEnvironmentCheckUtils uses the SDK's EC2MetadataUtils. That util class calls an EC2-specific metadata endpoint that is present on EC2 instances. ECS has its own set of metadata endpoints (https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-metadata-endpoint.html). One option would be to create a helper class to retrieve data from those endpoints and see if it comes back with any data. Another option would be to check for the presence of an ECS_CONTAINER_METADATA_URI environment variable and, if it's set, assume we're running in ECS and isRunningOnCloudEnvironment should return true.