ec2instances.info icon indicating copy to clipboard operation
ec2instances.info copied to clipboard

instance.json

Open Name404Error opened this issue 3 years ago • 8 comments

I was trying to run the code locally and when I typed Invoke build it gave a error saying no instance.json. I was wondering what this file is and how do I get/generate it.

Name404Error avatar Jul 14 '22 19:07 Name404Error

Hi! Do you have AWS credentials you are using that are stored in your environment? Let me try this as well and make sure something didn't recently break.

EverettBerry avatar Jul 14 '22 19:07 EverettBerry

I do not, I saw the "Running locally" section in the readme and wasn't sure where I need to put that info.

Name404Error avatar Jul 14 '22 19:07 Name404Error

Got it! There are 2 options to do this, one with docker and one on your machine. I personally use Docker but have plans to improve the devex for building natively as well.

  • Docker
docker build --build-arg AWS_ACCESS_KEY_ID=YOURKEY --build-arg AWS_SECRET_ACCESS_KEY=YOURKEY -t ec2instances.info .
  • Native
export AWS_ACCESS_KEY=YOURKEY
export AWS_SECRET_ACCESS_KEY=YOURKEY

P.S. One tip on running docker to develop locally. First run it (notice the http envvar and the volume)

docker run -it --rm --name ec2instances -v $(pwd):/opt/app --env HTTP_HOST=0.0.0.0 -p 8080:8080 ec2instances.info

Then jump into it:

docker exec -it ec2instances /bin/bash

EverettBerry avatar Jul 14 '22 19:07 EverettBerry

Thank you for the help. Now, when running invoke build, I get

botocore.exceptions.NoCredentialsError: Unable to locate credentials None

Then the code just stops, were the 2 access key from before not the credentials?

Name404Error avatar Jul 14 '22 22:07 Name404Error

I'm also running on windows if that changes things

Name404Error avatar Jul 14 '22 22:07 Name404Error

Ok that means the credentials are not being saved or loaded - please see here for information on how to set this for every environment, https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html

EverettBerry avatar Jul 14 '22 22:07 EverettBerry

when I do echo %AWS_ACCESS_KEY% or set AWS_ACCESS_KEY I see my key in the env, doesn't that mean it is saved/loadable?

my whole procedure atm is cd ec2instances.info/ python -m venv env env\Scripts\activate pip install -r requirements.txt set AWS_ACCESS_KEY=my key set AWS_SECRET_ACCESS_KEY=the other key invoke build Then it gives "botocore.exceptions.NoCredentialsError: Unable to locate credentials"

Name404Error avatar Jul 14 '22 23:07 Name404Error

Got it. I actually have a windows VM sitting around so let me spin that up and get back to you here.

EverettBerry avatar Jul 14 '22 23:07 EverettBerry

Same problem on Mac, tried both build-arg and env variable

Elentary avatar Jan 23 '23 23:01 Elentary

@Name404Error @Elentary the two envvars you need are this:

export AWS_ACCESS_KEY_ID=<your key>
export AWS_SECRET_ACCESS_KEY=<your key>

These can also be set in the ~/.aws/credentials file like so:

[default]
aws_access_key_id = <your key>
aws_secret_access_key = <your key>

Similarly, these can be passed to a docker environment like this:

docker build --no-cache --build-arg AWS_ACCESS_KEY_ID=<your key> --build-arg AWS_SECRET_ACCESS_KEY=<your key> -t ec2instances.info .

If none of these options are working for you, you can always download the instances.json file for any of the services directly from the website:

curl https://instances.vantage.sh/instances.json -o instances.json

EverettBerry avatar Jan 24 '23 08:01 EverettBerry

I'm experiencing the same issue trying to spin up locally on Mac.

Here are my steps so far:

  1. Clone the repo
  2. cd into repo
  3. run docker build --build-arg AWS_ACCESS_KEY_ID=<key id> --build-arg AWS_SECRET_ACCESS_KEY=<key> -t ec2instances.info .
  4. Fail with exception FileNotFoundError: [Errno 2] No such file or directory: 'www/cache/instances.json'. I've even tried downloading instances.json and placing it into 'ec2instances.info/www/cache/' and I still get the error.

Could this be something to do with the permissions of the AWS User? I've created a new IAM user whose credentials I'm using with the following permissions: { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "ec2:DescribeInstanceTypes", "ec2:DescribeRegions", "pricing:*" ], "Resource": "*" } ] }

Perhaps there's a permissions set or other AWS setup that I'm missing?

aprice-dev avatar May 09 '23 17:05 aprice-dev

That's almost certainly what's happening. In the log output you should see an Access Denied error but here is the boto3 call that cache.py uses https://github.com/vantage-sh/ec2instances.info/blob/master/cache.py#L337

EverettBerry avatar May 09 '23 17:05 EverettBerry

@EverettBerry Is there an example of a permission set for the IAM user that will allow spinning up localdev? The project README.md says all you need to run locally are the permissions I listed above, but that doesn't seem to actually cover what's required.

Alternatively maybe I can remove elasticache from my local instance since I don't need that currently for my use case.

aprice-dev avatar May 09 '23 18:05 aprice-dev

I believe the permission would be: elasticache:DescribeEngineDefaultParameters. It would be great if you could check that actually. I can update the README accordingly.

EverettBerry avatar May 09 '23 19:05 EverettBerry

I also created this pull request a while back which should make it even easier to run locally using docker-compose, which is shipped as part of the Docker application on macOS.

https://github.com/vantage-sh/ec2instances.info/pull/685

cristim avatar May 11 '23 07:05 cristim

@cristim sorry for the delay here. I'm planning to merge these changes this month.

EverettBerry avatar Jun 05 '23 20:06 EverettBerry

Believe this was resolved in #713

EverettBerry avatar Jan 12 '24 14:01 EverettBerry