cb-spider icon indicating copy to clipboard operation
cb-spider copied to clipboard
verified publisher icon cloud-barista

Error in the first attempt for `regionzone` API with a given AWS connection

Open seokho-son opened this issue 1 year ago • 10 comments

What happened

  • regionzone API를 AWS 에 대해 실행시, 내부 오류 발생
    • 모든 리스트에 대해 에러가 발생하는 것으로 보이나, 결과는 200 성공으로 빈 어레이를 리턴함.
    • Azure, GCP는 정상 동작 확인.

How to reproduce it (as minimally and precisely as possible)

  • 해당 API를 실행해본적이 없는 계정들로 시도

Anything else we need to know?

  • Spider 에러 로그 일부
[CB-SPIDER].[ERROR]: 2024-01-17 06:55:05 RegionZoneHandler.go:44, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.(*AwsRegionZoneHandler).ListRegionZone.func1() - AuthFailure on [ap-south-2] 
[CB-SPIDER].[ERROR]: 2024-01-17 06:55:05 RegionZoneHandler.go:45, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.(*AwsRegionZoneHandler).ListRegionZone.func1() - NoCredentialProviders: no valid providers in chain. Deprecated.
        For verbose messaging see aws.Config.CredentialsChainVerboseErrors 
[HISCALL].[129.254.175.187] 2024-01-17 06:55:05 (Wednesday) github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.DescribeAvailabilityZones():764 - "CloudOS" : "AWS", "RegionZone" : "", "ResourceType" : "REGIONZONE", "ResourceName" : "", "CloudOSAPI" : "DescribeAvailabilityZones()", "ElapsedTime" : "6.3058", "ErrorMSG" : ""
[CB-SPIDER].[ERROR]: 2024-01-17 06:55:05 CommonHandler.go:767, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.DescribeAvailabilityZones() - NoCredentialProviders: no valid providers in chain. Deprecated.
        For verbose messaging see aws.Config.CredentialsChainVerboseErrors 
[CB-SPIDER].[ERROR]: 2024-01-17 06:55:05 RegionZoneHandler.go:44, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.(*AwsRegionZoneHandler).ListRegionZone.func1() - AuthFailure on [me-central-1] 
[CB-SPIDER].[ERROR]: 2024-01-17 06:55:05 RegionZoneHandler.go:45, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.(*AwsRegionZoneHandler).ListRegionZone.func1() - NoCredentialProviders: no valid providers in chain. Deprecated.
        For verbose messaging see aws.Config.CredentialsChainVerboseErrors 
[HISCALL].[129.254.175.187] 2024-01-17 06:55:05 (Wednesday) github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.DescribeAvailabilityZones():764 - "CloudOS" : "AWS", "RegionZone" : "", "ResourceType" : "REGIONZONE", "ResourceName" : "", "CloudOSAPI" : "DescribeAvailabilityZones()", "ElapsedTime" : "6.3080", "ErrorMSG" : ""
[CB-SPIDER].[ERROR]: 2024-01-17 06:55:05 CommonHandler.go:767, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.DescribeAvailabilityZones() - NoCredentialProviders: no valid providers in chain. Deprecated.
        For verbose messaging see aws.Config.CredentialsChainVerboseErrors 
[CB-SPIDER].[ERROR]: 2024-01-17 06:55:05 RegionZoneHandler.go:44, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.(*AwsRegionZoneHandler).ListRegionZone.func1() - AuthFailure on [eu-west-1] 
[CB-SPIDER].[ERROR]: 2024-01-17 06:55:05 RegionZoneHandler.go:45, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.(*AwsRegionZoneHandler).ListRegionZone.func1() - NoCredentialProviders: no valid providers in chain. Deprecated.
        For verbose messaging see aws.Config.CredentialsChainVerboseErrors 
[HISCALL].[129.254.175.187] 2024-01-17 06:55:05 (Wednesday) github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.DescribeAvailabilityZones():764 - "CloudOS" : "AWS", "RegionZone" : "", "ResourceType" : "REGIONZONE", "ResourceName" : "", "CloudOSAPI" : "DescribeAvailabilityZones()", "ElapsedTime" : "6.3235", "ErrorMSG" : ""
[HISCALL].[129.254.175.187] 2024-01-17 06:55:05 (Wednesday) github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.DescribeAvailabilityZones():764 - "CloudOS" : "AWS", "RegionZone" : "", "ResourceType" : "REGIONZONE", "ResourceName" : "", "CloudOSAPI" : "DescribeAvailabilityZones()", "ElapsedTime" : "6.3238", "ErrorMSG" : ""
[CB-SPIDER].[ERROR]: 2024-01-17 06:55:05 CommonHandler.go:767, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.DescribeAvailabilityZones() - NoCredentialProviders: no valid providers in chain. Deprecated.
        For verbose messaging see aws.Config.CredentialsChainVerboseErrors 
[CB-SPIDER].[ERROR]: 2024-01-17 06:55:05 CommonHandler.go:767, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.DescribeAvailabilityZones() - NoCredentialProviders: no valid providers in chain. Deprecated.
        For verbose messaging see aws.Config.CredentialsChainVerboseErrors 
[HISCALL].[129.254.175.187] 2024-01-17 06:55:05 (Wednesday) github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.DescribeAvailabilityZones():764 - "CloudOS" : "AWS", "RegionZone" : "", "ResourceType" : "REGIONZONE", "ResourceName" : "", "CloudOSAPI" : "DescribeAvailabilityZones()", "ElapsedTime" : "6.3241", "ErrorMSG" : ""
[CB-SPIDER].[ERROR]: 2024-01-17 06:55:05 CommonHandler.go:767, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.DescribeAvailabilityZones() - NoCredentialProviders: no valid providers in chain. Deprecated.
        For verbose messaging see aws.Config.CredentialsChainVerboseErrors 
[CB-SPIDER].[ERROR]: 2024-01-17 06:55:05 RegionZoneHandler.go:44, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.(*AwsRegionZoneHandler).ListRegionZone.func1() - AuthFailure on [ap-northeast-1] 
[CB-SPIDER].[ERROR]: 2024-01-17 06:55:05 RegionZoneHandler.go:45, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.(*AwsRegionZoneHandler).ListRegionZone.func1() - NoCredentialProviders: no valid providers in chain. Deprecated.
        For verbose messaging see aws.Config.CredentialsChainVerboseErrors 
[CB-SPIDER].[ERROR]: 2024-01-17 06:55:05 RegionZoneHandler.go:44, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.(*AwsRegionZoneHandler).ListRegionZone.func1() - AuthFailure on [eu-south-1] 
[CB-SPIDER].[ERROR]: 2024-01-17 06:55:05 RegionZoneHandler.go:45, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.(*AwsRegionZoneHandler).ListRegionZone.func1() - NoCredentialProviders: no valid providers in chain. Deprecated.
        For verbose messaging see aws.Config.CredentialsChainVerboseErrors 
[CB-SPIDER].[ERROR]: 2024-01-17 06:55:05 RegionZoneHandler.go:44, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.(*AwsRegionZoneHandler).ListRegionZone.func1() - AuthFailure on [us-east-1] 
[CB-SPIDER].[ERROR]: 2024-01-17 06:55:05 RegionZoneHandler.go:45, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.(*AwsRegionZoneHandler).ListRegionZone.func1() - NoCredentialProviders: no valid providers in chain. Deprecated.
        For verbose messaging see aws.Config.CredentialsChainVerboseErrors 
[HISCALL].[129.254.175.187] 2024-01-17 06:55:05 (Wednesday) github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.DescribeAvailabilityZones():764 - "CloudOS" : "AWS", "RegionZone" : "", "ResourceType" : "REGIONZONE", "ResourceName" : "", "CloudOSAPI" : "DescribeAvailabilityZones()", "ElapsedTime" : "6.3316", "ErrorMSG" : ""
[CB-SPIDER].[ERROR]: 2024-01-17 06:55:05 CommonHandler.go:767, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.DescribeAvailabilityZones() - NoCredentialProviders: no valid providers in chain. Deprecated.
        For verbose messaging see aws.Config.CredentialsChainVerboseErrors 
[CB-SPIDER].[ERROR]: 2024-01-17 06:55:05 RegionZoneHandler.go:44, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.(*AwsRegionZoneHandler).ListRegionZone.func1() - AuthFailure on [ap-northeast-3] 
[CB-SPIDER].[ERROR]: 2024-01-17 06:55:05 RegionZoneHandler.go:45, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.(*AwsRegionZoneHandler).ListRegionZone.func1() - NoCredentialProviders: no valid providers in chain. Deprecated.
        For verbose messaging see aws.Config.CredentialsChainVerboseErrors

[CB-SPIDER].[ERROR]: 2024-01-17 06:55:05 RegionZoneHandler.go:44, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.(*AwsRegionZoneHandler).ListRegionZone.func1() - AuthFailure on [ap-south-2] [CB-SPIDER].[ERROR]: 2024-01-17 06:55:05 RegionZoneHandler.go:45, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/aws/resources.(*AwsRegionZoneHandler).ListRegionZone.func1() - NoCredentialProviders: no valid providers in chain. Deprecated.

Environment

  • Source version or branch: 0.8.1 릴리스
  • OS: Ubuntu 18.04
  • Others:

Proposed solution

  • 계정에 추가적인 롤바인딩이 필요하다면, 지침 및 방법 가이드 필요
  • 에러 메시지 등, 사용자에게 오류 전달 방법 개선 필요

seokho-son avatar Jan 17 '24 07:01 seokho-son

@raccoon-mh

  • 필요 role 등 확인 부탁 드립니다.
  • 특이사항 존재시 현황 공유 및 다음 링크에 추가 부탁 드립니다.

powerkimhub avatar Jan 17 '24 07:01 powerkimhub

@powerkimhub @seokho-son

  • 다음 AWS Price IAM 권한 관련 Docs 첨부 드립니다. https://docs.aws.amazon.com/ko_kr/awsaccountbilling/latest/aboutv2/using-price-list-query-api.html#iam-permissions-for-price-list-query-api

  • 첨부해주신 문서에 대한 수정권한 요청드립니다. 권한 주시면 관련 특이사항 추가하겠습니다.

raccoon-mh avatar Jan 23 '24 04:01 raccoon-mh

@raccoon-mh 감사합니다. 그리고, 권한 부족인 경우 오류로 리턴해주시는 것이 더 좋을 것 같습니다. :)

seokho-son avatar Jan 23 '24 04:01 seokho-son

넵. 이번 통합 테스트 이후 드라이버 보완에 포함하여 PR 드리겠습니다.

raccoon-mh avatar Jan 23 '24 04:01 raccoon-mh

@powerkimhub

  • 공유시트 권한 상승 감사합니다.
  • CSP별 Region.Zone 제공 현황 분석 요약시트에 특이사항으로 해당 링크와 IAM 권한 요구사항 추가하였습니다.
  • 추후에 통합테스트 관련 보완하여 본 이슈 완료하겠습니다.

raccoon-mh avatar Jan 24 '24 00:01 raccoon-mh

  • 넵, 캄사합니다~

powerkimhub avatar Jan 24 '24 00:01 powerkimhub

@raccoon-mh (@powerkimhub)

AWS region/zone 조회가 되지 않아서 이슈를 열었고,

말씀하신 IAM을 추가해봤는데, 아무래도 region이 아니라 price 에 대한 정보를 주신 것 같습니다. 링크 https://docs.aws.amazon.com/ko_kr/awsaccountbilling/latest/aboutv2/using-price-list-query-api.html#iam-permissions-for-price-list-query-api

혹시나해서, pricing 권한도 모두 추가해보았는데, region 조회는 되지 않는 것 같습니다. "Statement": [ { "Action": [ "pricing:*" ], "Effect": "Allow", "Resource": "*" } ]

https://docs.aws.amazon.com/accounts/latest/reference/API_ListRegions.html#API_ListRegions_RequestBody 가 리전 조회에 관련된 권한이 아닐까 싶네요. (organization's management account or a delegated administrator account.)

seokho-son avatar Feb 29 '24 08:02 seokho-son

@raccoon-mh @seokho-son


  • 현재 오류는 AWS SDK 인증 방법의 차이로 발생하는 오류 입니다.

    • [VPC~Cluster 등 기존 Driver 개발 방식] AWS SDK API 인자로 넘기는 방식
    • [RegionZone, PriceInfo 신규 Driver 개발 방식] ~/.aws/credential 설정 값을 활용하는 방식
      • 코드 확인은 안해봤지만, 시험 결과로 확인한 상황 입니다.

  • RegionZone, PriceInfo 기능 동작

    • 개발/실행 환경에 ~/.aws/credential 존재시 정상 동작
    • Container 실행 환경과 같이 ~/.aws/credential 없는 경우 현재 오류 발생

[@raccoon-mh]

  • 기존 방식과 동일하게 인자 방식으로 SDK 인증 방법 정정 부탁드립니다.

[@seokho-son]

  • patch를 기다리시거나,
  • spider container 실행 명령에 다음 인자 추가(단, local 환경에 crdential 파일 존재시)
    • -v ~/.aws:/root/.aws

powerkimhub avatar Feb 29 '24 11:02 powerkimhub

현황 파악 감사합니다. 급하게 필요하지는 않아서, patch를 기다리도록 하겠습니다.

seokho-son avatar Feb 29 '24 11:02 seokho-son

기존 credential 이용 방식과 STS 인증 방식 테스트하며 반영토록하겠습니다. 감사합니다.

raccoon-mh avatar Feb 29 '24 14:02 raccoon-mh

  • Completed via #1294

powerkimhub avatar Aug 23 '24 06:08 powerkimhub