awless icon indicating copy to clipboard operation
awless copied to clipboard

ec2-price.com down breaks pricer

Open boldandbusted opened this issue 6 years ago • 7 comments

 awless inspect -i pricer
[info]    Running full sync before inspection (disable it with --local flag)

Fetching prices at http://ec2-price.com for region global

fetching price for 'm1.xlarge': strconv.ParseFloat: parsing "<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>5
02 Bad Gateway</h1></center>\r\n<hr><center>nginx/1.6.2</center>\r\n</body>\r\n</html>\r\n": invalid syntax
fetching price for 't2.micro': strconv.ParseFloat: parsing "<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>50
2 Bad Gateway</h1></center>\r\n<hr><center>nginx/1.6.2</center>\r\n</body>\r\n</html>\r\n": invalid syntax
fetching price for 'm4.large': strconv.ParseFloat: parsing "<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>50
2 Bad Gateway</h1></center>\r\n<hr><center>nginx/1.6.2</center>\r\n</body>\r\n</html>\r\n": invalid syntax
fetching price for 't2.medium': strconv.ParseFloat: parsing "<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>5
02 Bad Gateway</h1></center>\r\n<hr><center>nginx/1.6.2</center>\r\n</body>\r\n</html>\r\n": invalid syntax
fetching price for 't2.small': strconv.ParseFloat: parsing "<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>50
2 Bad Gateway</h1></center>\r\n<hr><center>nginx/1.6.2</center>\r\n</body>\r\n</html>\r\n": invalid syntax
fetching price for 't2.large': strconv.ParseFloat: parsing "<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>50
2 Bad Gateway</h1></center>\r\n<hr><center>nginx/1.6.2</center>\r\n</body>\r\n</html>\r\n": invalid syntax
fetching price for 'm4.2xlarge': strconv.ParseFloat: parsing "<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>502 Bad Gateway</h1></center>\r\n<hr><center>nginx/1.6.2</center>\r\n</body>\r\n</html>\r\n": invalid syntax
fetching price for 'm4.xlarge': strconv.ParseFloat: parsing "<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>5
02 Bad Gateway</h1></center>\r\n<hr><center>nginx/1.6.2</center>\r\n</body>\r\n</html>\r\n": invalid syntax
Instance        Count   Estimated total/day (no EBS)
--------        -----   ----------------------------
m4.2xlarge      2
m4.large        17
t2.micro        15
t2.medium       45
m4.xlarge       4
t2.small        8
t2.large        2
m1.xlarge       1
                        $0.00

Dunno if this is maintained by the project - apologies if it is not and there's nothing that can be done. Thanks for awless! :)

boldandbusted avatar May 16 '18 19:05 boldandbusted

Hi @boldandbusted , thanks for reporting.

Unfortunately the API we use in the inspector (http://ec2-price.com/) is out of service now. All the code for the pricer is basically located here

Hopefully we could switch to a better service, but we have not find one yet. Other solutions are feasible (ex: caching prices locally and using AWS API to get prices, etc.) also but we are not focusing on that at the moment or in the near future.

Although this could be a good contribution for someone who has the time since inspectors are quite independent.

Thanks for your support!

simcap avatar May 22 '18 11:05 simcap

I would also love to have this feature back, but am doubtful that I could replace the existing dependency on ec2-price.

I use https://ec2instances.info for quick lookups, but it looks like all the pricing info that is needed should be in https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/index.json, however that's a 50mb file and somewhat awkward to parse through.

Theres a few places that talk about parsing that file in python though, not sure if it would be much harder in go: https://forrestbrazeal.com/2016/09/25/adventures-in-aws-understanding-the-price-list-api

adzuci avatar Jun 21 '18 03:06 adzuci

Hi, We knew that pricing information are available in JSON on AWS servers, but the problem is that we don't want to download and parse 50Mb JSON each time we launch this inspector. That's why we rely(ed) on ec2-price to get the data. The solution would have been to expose a service fetching these data from AWS and exposing them with a more friendly API. But we don't have the time/will to do that. If someone expose such an API, we would be happy to migrate from ec2-price to such an API. Another (less satisfactory) solution would be to download in the cache folder of awless ~/.awless/cache these big JSON files and update them periodically and use them when launching the inspector.

ps: Parsing JSON files in go would not be a problem as it is one basis of the standard library.

fxaguessy avatar Jun 21 '18 07:06 fxaguessy

Oh, that is newer. Can we user some filters in prices query?

aleksei@myhost$ `awless -p PROD inspect -i pricer --local Fetching prices at http://ec2-price.com for region global

fetching price for 'c4.large': strconv.ParseFloat: parsing "\r\n

502 Bad Gateway\r\n<body bgcolor="white">\r\n

502 Bad Gateway

\r\n
nginx/1.6.2
\r\n\r\n

\r\n": invalid syntax fetching price for 't2.small': strconv.ParseFloat: parsing "\r\n

502 Bad Gateway

\r\n<body bgcolor="white">\r\n

502 Bad Gateway

\r\n


nginx/1.6.2

\r\n\r\n

\r\n": invalid syntax fetching price for 'c4.xlarge': strconv.ParseFloat: parsing "\r\n

502 Bad Gateway

\r\n<body bgcolor="white">\r\n

502 Bad Gateway

\r\n


nginx/1.6.2

\r\n\r\n

\r\n": invalid syntax fetching price for 't2.micro': strconv.ParseFloat: parsing "\r\n

502 Bad Gateway

\r\n<body bgcolor="white">\r\n

502 Bad Gateway

\r\n


nginx/1.6.2

\r\n\r\n

\r\n": invalid syntax fetching price for 'c5.large': strconv.ParseFloat: parsing "\r\n

502 Bad Gateway

\r\n<body bgcolor="white">\r\n

502 Bad Gateway

\r\n


nginx/1.6.2

\r\n\r\n

\r\n": invalid syntax fetching price for 'm3.medium': strconv.ParseFloat: parsing "\r\n

502 Bad Gateway

\r\n<body bgcolor="white">\r\n

502 Bad Gateway

\r\n


nginx/1.6.2

\r\n\r\n

\r\n": invalid syntax fetching price for 'c5.xlarge': strconv.ParseFloat: parsing "\r\n

502 Bad Gateway

\r\n<body bgcolor="white">\r\n

502 Bad Gateway

\r\n


nginx/1.6.2

\r\n\r\n

\r\n": invalid syntax fetching price for 't2.medium': strconv.ParseFloat: parsing "\r\n

502 Bad Gateway

\r\n<body bgcolor="white">\r\n

502 Bad Gateway

\r\n


nginx/1.6.2

\r\n\r\n

\r\n": invalid syntax Instance Count Estimated total/day (no EBS)


t2.micro 4 t2.small 5 c4.large 1 c5.xlarge 2 c4.xlarge 1 m3.medium 1 t2.medium 24 c5.large 6 $0.00 aleksei@myhost:~$ awless version version=v0.1.11, commit=d6094ca2f507ce9ecd8c8f8de030fbea60923ebf, build-date=2018-06-21T12:04:05+02:00, build-arch=amd64, build-os=linux, build-for=targz `

Niemi avatar Nov 21 '18 11:11 Niemi

The pricer is not to be used anymore since the service http://ec2-price.com/ that the pricer used is no longer viable.

simcap avatar Nov 21 '18 11:11 simcap

What would be a nice pricing API for awless to consume? Implementing it might be a cool weekend project

dserodio avatar Jan 30 '19 13:01 dserodio

As a suggestion to fix this feature, we could use the ec2-instances.info scraper. https://github.com/powdahound/ec2instances.info/blob/master/scrape.py

danielhoherd avatar Dec 09 '19 20:12 danielhoherd