bottlerocket
bottlerocket copied to clipboard
bottlerocket-os
FIPS Support / Validation
This is related to my comment in #1297
Will you adding support for FIPS 140-2 standards? Please note: I am NOT asking about FIPS validation with regards to the containers themselves - I understand that is handled, at least technically speaking, during the build phase of the container image. FIPS 140-2 compliance for other OSes (Amazon Linux 2, RHEL) configures the crypto module to operate in compliance (proper crypto algorithms, entropy, etc) - these would be done in container image; however, there are other changes at the kernel-level that occur as well. I don't believe full on validation is in scope for Bottlerocket since crypto modules themselves get validated (not necessarily an OS) but perhaps guidance and/or documentation on meeting compliance requirements would be helpful.
A lack of compliance statement for CIS is preventing our company (and i'm sure many others) from adopting Bottlerocket.
Any updates around FIPS compliance and the usage of Bottlerocket OS? It's a blocker for our team to be able to use for Fedramp compliant EKS nodes. Any updates would be very much appreciated!
Hey @JohnTrevorBurke i believe I just added you in linked in. I may be able to help.
Hi Team, I have the same query here, I just want to confirm to know if AWS BottleRocket AMI's are FIPS-140-2 compliant by default or not?
Looking forward to your reply.
FIPS compliance is our second most requested feature, behind CIS (which is in progress), and I'm planning to focus on it once the CIS benchmark is complete.
There are (at least) two interpretations of what this means, and different auditors may have different takes:
- Bottlerocket's kernel uses compliant algorithms when
fips=1is passed on the kernel command line, and its userspace tries to arrange for containers to do the same (by mounting in/etc/system-fipsand such). - In addition to the above, Bottlerocket's kernel and userspace are themselves FIPS certified, meaning that they only use validated, FIPS-compliant crypto.
The first should be somewhat lower hanging fruit, and it's how I understand @vennemp's initial request. But I've also heard interest in the second, stronger version. That's a bigger lift because it likely means switching away from rustls and Go's crypto library, using OpenSSL for everything instead, and adopting something like Go FIPS rather than the regular Go toolchain.
Hi @bputt-e - the hope is someone can work on this in 2023, but there is no definite time set at this point.
Hi @stmcginnis wanted to check in on updates or any rough timeline for Bottlerocket FIPS 140-2 support in 2023?
Hi @amandagalligan! No timeline yet, but FIPS compliance is an important thing on the roadmap. Unfortunately it will take some time to get everything in place. But there is work being done towards that end. I just can't give any kind of time estimate at this point, unfortunately.
Would it make sense to publish an image containing only the first part mentioned by @bcressey above, since it is a low hanging fruit?
It is a good starting point for most customers - we'd still need to solve the user space library problem - which needs to be solved on a case by case basis anyway
Hi @filipenf - thanks for the comment. Like Ben mentioned, implementing fips=1 being passed to the kernel command line is pretty low hanging fruit. But publishing and distributing an image for Bottlerocket that supports that is a heavy lift on its own. And we want to make sure we're shipping the most complete and quality software we can.
Like Sean mentioned, there's work being done to make this happen since this is a very highly requested feature but there's no timeline yet.
Just a bump, we are also tracking this and it's FIPS compliance for FedRAMP purposes. Thanks for all you guys do!
Not much of an update, but current status is there are a few things in-flight towards getting FIPS certification, but it it will be some time before that can be completed.
Bottlerocket leverages the work done with the kernel from Amazon Linux. There is active work being done there to support FIPS. Then the certification process itself takes some time to go through. So I don't have a lot of concrete details, but with the work that needs to be done yet, it is looking like it won't be any sooner than some time in 2024.
Looks like AWS LC is nearing FIPS certification (https://csrc.nist.gov/Projects/cryptographic-module-validation-program/modules-in-process/Modules-In-Process-List) Will that help, or do you use AWS-LC Cryptographic Module v2.0 and Amazon Linux 2023, which are much farther behind in the certification process?
Thanks for the update. I am bumping because we are yet another org that needs FIPS for FedRAMP compliance. I was just about to go live in a few weeks with an EKS cluster using bottlerocket and was told today by the compliance team that we need to switch OS on our nodes because of this. So it is a full blocker for us.
No FIPS = No FedRamp = No Bottlerocket 😢
Hoping this gets resolved soon.
Just commenting to add some more clarity to this. If you are using containers, FIPS mode on the OS is not in scope. That only changes OS-level configurations. I was only asking for documentation from AWS speaking to this because I know the auditors and compliance folks don't know this.
The cryptomodules inside the container images are what need to be validated. If you are using bottle rocket to run non-container workloads, (can't imagine a reason why) that use the default OS module then it would need to be the FIPS validated module and then maybe FIPS mode would be in scope. Though you can always hard-wire this yourself.
Enabling FIPS mode on OSes only switches to a FIPS validated version of the module for default- like OpenSSL and disables non-compliant algorithms like SHA-1 for hashing. Sometimes they include logic in the boot sequence to change for known hashes of encrypted strings to test whether the crypto-module is running properly, if the hashes don't match, it reboots and tries again - these are the kernel changes I was referring in my previous message. But again, this has nothing to do with the container's cryptographic modules - this is where I was off-base previously. I know I am just a random guy on GitHub saying this, which is why I want the major cloud providers to speak to this officially.
@vennemp The OS is definitely in scope if the container uses the shared kernel for anything requiring a cryptographic operation. fips=1 must be passed to the linux kernel at boot time to ensure the kernel itself only utilizes appropriate cryptographic altos/etc. Additionally, the container crypto libs (openssl/etc.) must ALSO be FIPS. Tons of drivers do things like TLS offload or acceleration, etc. as an example. If I were validating, you'd have to prove to me your app/data at no point is exposed to kernel crypto to let it be marked out of scope. The host OS user-space modules I can see easily being out of scope with a containerized app, but not the kernel.
Are you aware of any container images that don't include their own cryptographic modules? If so, I would imagine their use is very limited; it could limit the container's portability between OSes and you'd have to mount a volume on the host, where the crypto libs and module reside, inside the container at runtime for it to be used. Your point is well taken though, in the unlikely event the container is using the host's modules, yes that would make the OS's FIPS status relevant for the application.
One small wrinkle I am aware of is the use of Cilium's Transparent Data Encryption to automatically encrypt node to node traffic. This uses the host's kernel implementation of IPSec. But the container that runs Cilium is not actually performing the encryption, it is making changes to the host's routing config to create the IPSec tunnels- which is a subtle but important difference. Since the host is actually encrypting the traffic then the FIPS status of the host is in scope when using Cilium's TDE. Any other application that works in a similar fashion would require FIPS support on host and would preclude you from using bottlerocket until this is addressed.
RedHat UBI 9 containers require FIPS mode to be enabled on the host OS and require a bind-mount of /etc/system-fips from the host into the container in order for the FIPS validation tests to pass successfully. (Despite having its own crypto libs in the container).
One fairly widespread example of a container dependency on host crypto is the kernel PRNG, accessible via /dev/random, /dev/urandom, or the getrandom() syscall. Pretty much every standard library or runtime for popular languages will default to these mechanisms for getting random bytes on Linux.
I've owed this issue a more substantial update for a while so here goes.
In the current plan, FIPS for Bottlerocket requires two modules:
- AWS-LC Cryptographic module (in "Coordination" state)
- Amazon Linux 2023 Kernel Crypto API Cryptographic Module (in "Implementation Under Test" state)
NIST doesn't share the expected timeline but based on historical data scraped from those links, it looks like validation may take up to 650 days - meaning December 2024 before the kernel will be validated.
The remaining implementation work required is:
- Create a golang build that swaps in aws-lc for the Go crypto module, using the upstream
GOEXPERIMENT=boringcrypto. - Figure out how to plug in aws-lc-rs instead of ring at build time.
- Adjust templates and helpers to automatically use FIPS API endpoints instead of non-FIPS
Given that FIPS entails some functional and cryptographic trade-offs - no Wireguard support in the kernel, none of the newer, well-regarded non-FIPS algorithms, more overhead in crypto operations in Go, etc - my plan is to release a parallel set of FIPS variants.
For example:
-
aws-k8s-1.28would be non-FIPS, use native Rust and Go crypto, and provide the full set of kernel functionality, same as today. -
aws-k8s-1.28-fipswould be FIPS enabled out of the box, useaws-lcbacked crypto, havefips=1on the kernel command line, mount/etc/system-fipsinto containers, and use FIPS API endpoints for communicating with EKS, ECR, and EC2.
Some exciting news from last week - AWS-LC is now FIPS 140-3 certified - so it's just the 6.1 kernel that's left.