dump247
*WIP* Docker support
- Adds Dockerfile to run on a linx host or VM in on a mac.
TODO
- [ ] Add back 2 factor auth somehow
- [ ] Add support for specifying the credentials profile boto uses
I think it's a good idea to remove the MFA token prompt functionality. I did that before I figured out how to generate the pass codes and it is a pretty hacky, platform-specific solution. Without the prompt, MFA should work fine with generated codes.
We're developing inside CoreOS VMs so run services as containers, with one aim being we don't need to install all the dependencies on a Mac (or what ever the host). As some of the services need to access AWS resources, it's easier to use aws-mock-metadata to hand out keys than to pass around enviroment variables everywhere. Then we only need our personal AWS keys in a single place on the host ~/.aws/credentials.
We run aws-mock-metadata on CoreOS too so everyone doesn't also have to install that as a dependency. Then we use iptables to intercept the calls as you do on a Mac. https://github.com/EqualMedia/coreos-vagrant/commit/4e1eff6d41a3ad44921a6db0194ba338ce392fc8
We run this container something like docker run -v ${HOME}/.aws:/root/.aws:ro -p 45000:45000 state_proto/aws-mock-metadata:latest
If that makes sense. It means our dev setup is also more consistent with production.
Sweet! I originally wrote this so the company I work for could enable two factor auth. This service makes that as painless as possible. I also have another recent service you may or may not find interesting: https://github.com/dump247/docker-ec2-metadata.
@dump247 re: your comment about removing the MFA token prompt -- is there some way for this to work on an MFA account without prompting for MFA?
Would be curious to try this docker solution but my personal creds are MFA.
You can put the seed of the TOTP in the config, then it can generate the codes for you (This isn't coded yet and would need adding). That partially defeats the point as then the seed and the auth tokens are all on the same machines. But might work for you.
Also I thought I should add we've been using this PR for the last 6 months without problems for many developers.
Any thoughts on how we should progress?
@matschaffer @danharvey Generating TOTP codes has already been implemented in master. There is a config option to provide the secret.
MFA is not required to use this service. This service checks if MFA is enabled on the account and then attempts to use the MFA token to generate credentials. If MFA was enabled in configuration, instead of occurring by default, it should resolve the issue with merging this PR.
@matschaffer If you need something to provide credentials to docker containers in EC2, I would recommend using Docker EC2 Metadata. With that service, you use the instance profile and roles instead of access/secret key. This means you don't need to distribute those secrets and your credentials get automatically rotated.
@dump247 I'm a little curious about why MFA is needed for this when my usual key/secret work without MFA on the same account. But granted, I haven't dug into the details of this service at all yet.
https://github.com/dump247/docker-ec2-metadata looks handy if you want to override the profile for a given container but that's not my case at the moment. Things in EC2 work fine thanks to the metadata endpoint being accessible from docker.
What I'm trying to handle is locally running docker containers (e.g., boot2docker) having access to AWS services via my personal keys at dev time.
Just because MFA is enabled on an account doesn't mean that the policy that allows your account access to the resources enforces MFA on all API calls, you can pick and choose.
@matschaffer that's how we're using this with Docker locally on our Mac's, I can publish the systemd unit / routes we've added to CoreOS for this to work. I think it was open but we may have closed the github repo.
The issues with MFA we also hit, we have MFA for the console but not API calls. So having that configurable would help. Would fix the issue in this PR so I'll take a look at that unless you were @dump247?
I just got back from vacation with no internet access (woohoo!).
@matschaffer Sorry if I wasn't clear, but MFA is not required for the service to operate. It is just enabled by default if you have MFA enabled on your account. The initial use case was to enable seamless MFA-enabled API access, so that is why it is the default. Currently, there is no way to disable this operation, but it should be pretty easy to add.
@danharvey I will implement that today or tomorrow. It will require some documentation updates.
Is it reasonable for the default to remain as it is, and assume you want MFA API access, or should be default be to only enable MFA if you specifically enable it?
I guess an ideal might be to have it default to no MFA prompts but with a clear error upon hitting an MFA protected API. No idea if that's feasible though, so just a basic way to toggle MFA prompting off would be handy.
I don't have a preference either way for it being default or not. I don't see any advantage or disadvantage to either. I guess as long as it's documented clearly about why some API calls will fail if you've setup a MFA policy then that's fine. But I'd hope people know what they've setup!
I'll also dig out the linux iptables rules I use on CoreOS in a VM to get this to work, and document how to use the docker file.
Any preference to docs in the git repo or wiki? I think the former would be easier then we can review it.
@matschaffer This service only provides credentials, so there is no way to know how the credentials will be used. Also, I don't believe there is a clear method for determining if the API is MFA protected since that is embedded in IAM rule conditions.
@danharvey Put the documentation in the git repo in a /docs/ directory.
Ahh... Fair point. Configurability with default to MFA enabled for MFA accounts sounds like the right move then.
On Tuesday, June 23, 2015, Cory Thomas [email protected] wrote:
@matschaffer https://github.com/matschaffer This service only provides credentials, so there is no way to know how the credentials will be used. Also, I don't believe there is a clear method for determining if the API is MFA protected since that is embedded in IAM rule conditions.
@danharvey https://github.com/danharvey Put the documentation in the git repo in a /docs/ directory.
— Reply to this email directly or view it on GitHub https://github.com/dump247/aws-mock-metadata/pull/3#issuecomment-114506679 .
-Mat
about.me/matschaffer
I just pushed a change to master so you can disable MFA in the config file.
Commit 4047552d25006d27dc7ce59a78c13afc6e5d8fe4