machine
machine copied to clipboard
docker
Add encrypt EBS option to amazonec2 driver
Description
There is not good way to add encryption to an already unencrypted EBS root volume in AWS. It is often required to have an ec2 instance with an encrypted EBS volume. The only reasonable solution is to create the encrypted volume on instance creation. Therefore to create a machine with EBS encryption enabled the option must be sent via docker-machine.
This patch adds an option to the amazonec2 driver that enables EBS encryption.
Related issue(s)
None
We need this also, our enterprise automatically performs a shutdown on any instance with a non-encrypted EBS.
@abarrafo I have been using the patched version to create the instances and then after that unpatched works to control the instance.
@abarrafo you can also enable encryption by default for all aws ebs volumes (https://aws.amazon.com/about-aws/whats-new/2019/05/with-a-single-setting-you-can-encrypt-all-new-amazon-ebs-volumes/)
@abarrafo you can also enable encryption by default for all aws ebs volumes (https://aws.amazon.com/about-aws/whats-new/2019/05/with-a-single-setting-you-can-encrypt-all-new-amazon-ebs-volumes/)
I'll bring it up to our Public Cloud team to implement, our engineers do not have authority to make account level changes.
@npalm all customer-facing AWS accounts at my place of work have this setting enabled, but we still find docker machine (which is used by Gitlab runner) is unable to start up ec2's without the encrypted parameter set. They are auto-killed.
This change looks like it would fix our issue - how do we go about getting it merged?
@dc2tom sorry I am not the maintainer of this repo, maybe you could make a PR to the gitlab maintained one (https://gitlab.com/gitlab-org/ci-cd/docker-machine). About the encryption. For me the runner is working fine with default encryption enabled on ec2. The ebs of the docker machine is encrypted with a KMS key.
@npalm I have got this working now. It turns out that our organisation has a policy in place whereby although default EBS encryption is enabled, service roles allowed to access the default kms key are defined on a per-account basis.
"arn:aws:iam::ACCOUNTNUMBER:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling",
"arn:aws:iam::ACCOUNTNUMBER:role/aws-service-role/spot.amazonaws.com/AWSServiceRoleForEC2Spot"
By default the account management terraform was allowing AWSServiceRoleForAutoScaling but not AWSServiceRoleForEC2Spot, so spot requests being made by GitlabRunner were not able to use the default encryption key. This meant spot requests failed with "bad parameters".
@lsbrillant ideally, we could do with the ability to provide the name of an existing KMS key to use to encrypt the docker machine worker volumes, as well as flagging the worker to encrypt its storage.
It quite rightly feels like a bit of a limitation that we can specify a custom kms key in our launch configuration for the "manager" but are only able to currently use the default kms key for the "workers".
Happy to raise this as a request for the gitlab fork of docker-machine, but it feels like this would be something useful for the main branch of docker-machine. If users wish to encrypt the storage of docker-machine managed EC2s, they probably want to be able to specify which key to use.
@dc2tom I tried to add the ability to specify a KMS key, however doing so required upgrading the version of the golang AWS SDK being used. This broke lots of other stuff in the project and so I gave up on that feature.
@dc2tom I tried to add the ability to specify a KMS key, however doing so required upgrading the version of the golang AWS SDK being used. This broke lots of other stuff in the project and so I gave up on that feature.
Thanks for trying 👍 Reading between the lines it sounds like Gitlab may be looking to move away from Docker machine in the near future so hopefully it won't be required. https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4338
@Michenux I initially tried doing that but the version of the AWS SDK used in docker-machine was too old and did not have that option. Upgrading the version of the SDK broke much of the existing functionality.
unsubscribe
2021-07-31 1:37 GMT+08:00, Lucien @.***>:
@Michenux I initially tried doing that but the version of the AWS SDK used in docker-machine was too old and did not have that option. Upgrading the version of the SDK broke much of the existing functionality.
-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/docker/machine/pull/4743#issuecomment-890048602
