tock
RFC: External dependencies in the Tock kernel
History
Tock's current policy on external dependencies is we do not allow them. From doc/Design:
Tock chooses to not use any external libraries for any of the crates in the kernel. This is done to promote safety, as auditing the Tock code only requires inspecting the code in the Tock repository. Tock tries to be very specific with its use of unsafe, and tries to ensure that when it is used it is clear as to why. With external dependencies it would be significantly more challenging to ensure that uses of unsafe are valid, particularly as external libraries evolve.
Our one exception so far is to vendor dependencies and track them in the Tock repository directly:
Tock's compromise has been to pull in specific portions of libraries into the libraries folder. This puts the library's source in the same repository, while keeping the library as a clearly separate crate. We do try to limit how often this happens.
We also have discussed making exceptions for individual boards, where only the board crate would have a dependency:
The core group talked about including external dependencies, and we generally agreed that containing them to a very specific crate (with /boards/ being a reasonable place to store it) that is documented and clear why that crate exists is a reasonable way to support rubble.
and for downloading dependencies:
It is fine for the crate to be downloaded even if it is not being used by the board that is being compiled. However, it should be the case that out-of-tree boards can depend on the Kernel (e.g. chips / capsules / kernel / libraries) without having to download the dependency.
Cryptography
Where Tock's "no dependency policy in the kernel" may hit a limit is concerning implementations of cryptographic functions. We do not want to repeat mistakes and implement our own cryptography functions. We then have three options:
- Do not use software cryptographic functions in the kernel. Instead, use only hardware implementations or include them only in userspace.
- Use external libraries, but vendor them into the tock repository. This allows us to better track the dependency tree of specific external libraries.
- Use external libraries via standard cargo mechanisms.
RFC
This thread serves as a place to discuss our options and devise a policy for dependencies going forward.
Note: I'm going to keep separate opinions in separate comments, so you can thumbs up/down them individually.
I think Tock's no-external-dependencies policy is too strict, and we should be somewhat more open to adding external dependencies. With a better policy, the discussion would've started as "which crypto library is the best choice considering support/maturity/featureset/minimality" rather than "should we even use crypto libraries".
Oh, and being open to using libraries from the embedded Rust working group may help to bridge the gap between our communities.
If we decide to use external libraries, I would prefer to pull them in via standard cargo mechanisms for two reasons:
- Vendored dependencies are more effort to maintain than crates.io dependencies, because you have to keep them updated (especially important for crypto libs).
- Pulling dependencies from crates.io gives downstream projects more options for dependency management, allowing e.g. them to share the same version of a library between the Tock kernel and their host side code.
- Use external libraries, but vendor them into the tock repository. This allows us to better track the dependency tree of specific external libraries.
A few downsides of this approach:
- We will not receive automated security warnings when vulnerabilities are discovered in the libraries we depend on
- The "cost" of updating these dependencies to newer versions becomes higher, since we have to actually PR in all the changes to the downstream library
- We bloat our git repository with a bunch of files from another repo (or we use submodules, which are even worse)
- Use external libraries via standard cargo mechanisms.
If we are going to go this route I think it is still important to see if we can limit the dependencies to boards/. I understand that the kernel may want to use these crypto libraries, but given that we are likely to always want using hardware implementations to be possible it probably makes sense for boards to be passing in a reference to the crypto implementation in use anyway. We could also consider creating a new top-level folder (external_libraries/?) which by convention is only depended on by board crates directly. This external_libraries directory could contain individual library + component crates for each external dependency to help relieve the problem of all the individual board crates depending on the same components crate.
I wanted to +1 @jrvanwhy 's comments. I agree with him that the no external dependencies policy is too strict.
I'm not saying we should, but other embedded operating systems (mbed for example) pull in external code for drivers and have a significantly larger number of supported features and boards. Again, not saying we should, but it might be worth discussing for some specific cases.
The ability to work with the embedded Rust community might also be very useful. Although again that would be on a case by case basis. For example some of the svd2rust work.
But I think crypto dependencies are the first ones to consider, as they are the ones we realistically can't do ourselves.
In terms of only supporting dependencies to boards I think that is limiting. For example look at the AES-CCM PR which uses the dependency in capsules (although the board actually enables this capsule in this case).
I understand that the kernel may want to use these crypto libraries
I think that, especially for the kernel crate, and perhaps for arch crates as well, the rule against external dependencies is perhaps most useful. Thus I would propose that for almost all use cases within these crates where external dependencies can be useful (e.g. crypto), we should (perhaps with investment of more effort) define interfaces (i.e. traits) which external dependencies can then be "plugged in" to by other crates, versus using these dependencies directly. This retains many of the useful properties of our current stance on external dependencies, while fostering reusability of the kernel, as well as independence of a particular single external dependency.
For dependencies where this absolutely does not work (such as traits proving macros, used internally within the kernel), I believe that perhaps vendoring or strict version are still necessitated.
I think the main reason why we have been unwilling to change our dependency policy in general is that tracking the dependencies of dependencies is untenable and lacks tooling support. And it doesn't matter how good of a job we do about reasoning about unsafe and our abstractions, if a library N levels deep uses unsafe incorrectly then the kernel is subject to those flaws. And, as that code is out-of-tree it is hard to find and audit. Since cargo provides no way to enforce this programmatically, it would have to be done manually.
I support including external dependencies for only cryptographic functions, and from only a list of specific sources we would define (perhaps just https://github.com/RustCrypto). I also support there being a single (or some small number) of very specific ways they can be included (e.g. only as dependencies from a list of specific crates, or through a new crate). I also would like the dependency tree of the pinned version printed and included in the readme, ideally with links, so we are being as transparent as possible as to where the source code Tock is compiling comes from.
From @hudson-ayers:
but given that we are likely to always want using hardware implementations to be possible it probably makes sense for boards to be passing in a reference to the crypto implementation in use anyway
From @lschuermann:
we should (perhaps with investment of more effort) define interfaces (i.e. traits) which external dependencies can then be "plugged in" to by other crates, versus using these dependencies directly. This retains many of the useful properties of our current stance on external dependencies, while fostering reusability of the kernel, as well as independence of a particular single external dependency.
I think both these comments point towards the same idea, something like "external dependencies can only be exposed to the kernel through a specific HIL". I like this idea, as it 1) promotes clarity and ease-of-understanding (i.e. external code is used through specific traits, and not arbitrarily throughout a crate) and 2) would promote "swapability" of a HW implementation should it exist for a specific platform.
From @hudson-ayers:
but given that we are likely to always want using hardware implementations to be possible it probably makes sense for boards to be passing in a reference to the crypto implementation in use anyway
From @lschuermann:
we should (perhaps with investment of more effort) define interfaces (i.e. traits) which external dependencies can then be "plugged in" to by other crates, versus using these dependencies directly. This retains many of the useful properties of our current stance on external dependencies, while fostering reusability of the kernel, as well as independence of a particular single external dependency.
I think both these comments point towards the same idea, something like "external dependencies can only be exposed to the kernel through a specific HIL". I like this idea, as it 1) promotes clarity and ease-of-understanding (i.e. external code is used through specific traits, and not arbitrarily throughout a crate) and 2) would promote "swapability" of a HW implementation should it exist for a specific platform.
I like this idea as well. I would like to note that this effectively constrains external dependencies to abstractions that are conducive to hardware acceleration, such as cryptography, error-correcting codes, FFTs, and compression. In theory we could have drivers but this seems unlikely, as we would need to resolve its own resources through Tock (e.g., pin configuration).
I'm comfortable with this constraint and I think it draws an effective boundary around when we might want external implementations (e.g., an FFT library). We could of course always relax it later if new needs arise, but this seems quite effective for now.
That being said, I think that a policy of allowing a certain class of dependencies should not be interpreted as all dependencies in this class would be allowed. We've seen that idiomatic Rust code tends to have size issues, and so between a simple, custom lean implementation of an algorithm (which isn't a potential vulnerability) and a big implementation pulled in a from a library, we might opt for the former. This policy change would mean that such a introducing a dependency is possible, not that dependencies can be freely added.
