finops-toolkit icon indicating copy to clipboard operation
finops-toolkit copied to clipboard

[Hubs] Document how to bring in non-azure cost data (AWS, GCP) into FinOps Hub

Open DUeffing opened this issue 1 year ago • 6 comments

📝 Scenario Customers want to leverage the FinOps Hub solution to ingest cost data (FOCUS 1.0) from other cloud providers like GCP or AWS. Today the version 0.4 does not automatically process the GCP data, as the ADF pipeline run fails.

💎 Solution Customer want to have an overarching view of cost data across different cloud providers

🙋‍♀️ Ask for the community We could use your help:

  1. Please vote this issue up (👍) to prioritize it.
  2. Leave comments to help us solidify the vision.

DUeffing avatar Aug 20 '24 06:08 DUeffing

@allcontributors Please add DUeffing for ideas

flanakin avatar Aug 23 '24 03:08 flanakin

@flanakin

I've put up a pull request to add @DUeffing! :tada:

allcontributors[bot] avatar Aug 23 '24 03:08 allcontributors[bot]

[celebrate] Daniel Ueffing reacted to your message:


From: allcontributors[bot] @.> Sent: Friday, August 23, 2024 3:48:39 AM To: microsoft/finops-toolkit @.> Cc: Mention @.>; Author @.> Subject: Re: [microsoft/finops-toolkit] [Hubs] Document how to bring in non-azure cost data (AWS, GCP) into FinOps Hub (Issue #910)

@flanakinhttps://github.com/flanakin

I've put up a pull requesthttps://github.com/microsoft/finops-toolkit/pull/922 to add @DUeffinghttps://github.com/DUeffing! 🎉

— Reply to this email directly, view it on GitHubhttps://github.com/microsoft/finops-toolkit/issues/910#issuecomment-2306126135 or unsubscribehttps://github.com/notifications/unsubscribe-auth/AWUU5RLIOD4BISTPPCLYS4LZS2WJPBFKMF2HI4TJMJ2XIZLTSWBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDUOJ2WLJDOMFWWLLTXMF2GG2C7MFRXI2LWNF2HTAVFOZQWY5LFUVUXG43VMWSG4YLNMWVXI2DSMVQWIX3UPFYGLAVFOZQWY5LFVI2DSOJXGU3DEOJRG6SG4YLNMWUWQYLTL5WGCYTFNSBKK5TBNR2WLKRVGQ2TSOBVGA3TAMNENZQW2ZNJNBQXGX3MMFRGK3FMON2WE2TFMN2F65DZOBS2YSLTON2WKQ3PNVWWK3TUUZ2G64DJMNZZJAVEOR4XAZNKOJSXA33TNF2G64TZUV3GC3DVMWUTKOBXHE3DKOBWGKBKI5DZOBS2K2LTON2WLJLWMFWHKZNKGI2DONBYGYZTINRZQKSHI6LQMWSWYYLCMVWKK5TBNR2WLKRUHE4TONJWGI4TCN4CUR2HS4DFUVWGCYTFNSSXMYLMOVS2UNJUGU4TQNJQG4YDDJ3UOJUWOZ3FOKTGG4TFMF2GK. You are receiving this email because you were mentioned.

Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

DUeffing avatar Aug 23 '24 06:08 DUeffing

Please confirm if I can directly bring in focus data from aws s3 into finops hub

saleemmy avatar Oct 08 '24 18:10 saleemmy

@flanakin , please can you share the steps to bring in AWS focus data into the finops hub.

saleemmy avatar Oct 09 '24 17:10 saleemmy

Hi @saleemmy , out of the box the FinOps Hub toolkit is not capable to process AWS cost data in FOCUS format for the following reasons.

  • AWS provides as part of the export as well a .manifest file, however the content and structure totally differs from the one Azure creates
  • The leveraged FOCUS data field differ in some aspects between AWS and Azure (e. .g AWS provides the "AvailabilityZone" field and Azure not)
  • some fields are provided as type of "map" where in Azure all field are provided as string

Due to this, the standardly deployed ADF pipeline cannot process the AWS data.

As result a separate pipeline must be added manually to the FinOps Hub considering the above mentioned aspects as a custom solution

DUeffing avatar Oct 09 '24 18:10 DUeffing