image-spec
image-spec copied to clipboard
opencontainers
Name Delegation Design Discussion
DNS delegation is the idea that I can own the domain example.com but delegate hosting of my container images to container.hosting. Think MX records, where I have a DNS name that tells me where to send email for a domain without the dedicated special DNS record.
Today, AppC supports a method for doing this type of delegation that uses HTTPS, and HTML meta tags. This type of delegation is inspired by the method used by the go get package management system as well.
One critique of this method is that there is an RFC for this sort of meta information called 'well-known URIs'. I believe that using this type of well-known scheme is a better choice today and will stay out of people's way better; in fact we prototyped this in a project called abd.
FAQ Why not DNS: If we used DNS it would make it hard to bootstrap trust of this source based on the existing well understood TLS infrastructure on the internet today. With systems like Let's Encrypt in place today I think it would be smart to rely on this existing trust root. We do this in rkt today and it works nicely.
But I have this way better way of doing delegation: We can add additional methods, but having some method to start for delegation is super helpful.
I don't want to allow outbound traffic to the internet: Container engines can allow for this feature to be turned off or configure that certain domains are statically delegated to internal mirrors. This is all about having a default UX.
On Wed, Apr 06, 2016 at 05:15:43PM -0700, Brandon Philips wrote:
I believe that using this type of well-known scheme is a better choice today and will stay out of people's way better…
I think blob delegation should be outside the scope of the image format, since blobs that are content-addressed based on a cryptographic hash can be fetched from any content-addressable storage (CAS)—although registries probably want to include hints about hosts providing blobs required by a given image 1. Baking CAS repository information into the image manifests requires folks to make distribution-time decisions at image-creation time (and image authors and distributors need not be the same people).
Subject: DNS-based delegation (was: Proposal for a new project: OCI Image Format Spec)
Date: Sat, 12 Mar 2016 15:34:11 -0800
Message-ID: <[email protected]>
@wking This isn't about blob delegation. This is about delegation of naming for images. Which is explicitly in scope (see the README).
On Thu, Apr 07, 2016 at 06:47:35AM -0700, Brandon Philips wrote:
@wking This isn't about blob delegation. This is about delegation of naming for images. Which is explicitly in scope (see the README).
Ah, it's just about discovering manifest hashes. Developing and versioning the separate image/naming/signing components 1 in a single repository sounds complicated to me (do you make a major bump when you adjust the naming spec, even though you haven't touched the image spec?). But discovering manifest hashes using an approach like this sounds like one of several reasonable registry approaches to me.
@philips Let's make a note here to consider cases such as notary for naming delegation.
Have we decomposed naming delegation to particular set operations?
@stevvooe I don't see how notary relates to name delegation. Do you have a doc?
@philips You can read here. The link is fairly straightforward. Let me know if you have more questions.
Perhaps, I am misunderstanding naming delegation. We may need a clear definition and identification specific use cases it solves for the individual user. Part of the crux with naming delegation is that it is great for ISVs and service providers but I'd like to better understand where the user gains benefit. We struggled with our efforts from this angle when we looked into this. If I were, say, a data scientist, why should I have to setup DNS and a web service to deploy software in my local infrastructure?
I do find such strategies as go get or well-known urls to be elegant but getting them reliably deployed can be a nightmare in some organizations (well-known being the clear winner here, perhaps).
We also may want to value offline use cases slightly higher. In typical deployments, there may only be a few organizational mirrors. In the abd examples and some of the proposals we had in this area, it focuses on mirrors in the Internet at large but it seems more likely to have a local mirror, rather than spreading requests across the internet.
Every enterprise I work with wants a trusted repository of data. This is a model they are very familiar with. Think rpm/deb repo or Java's maven repos. How does this name delegation model work for this model? For example imagine nginx comes from nginx.com. But in the "enterprise" they want to mirror the images from nginx.com to their local servers such that when a user pulls "nginx" it goes to the on prem repo and not the internet.
Configuration to declare where the mirror is for nginx.com or more likely '*'. It is just namespaces that has some default useful semantics.
On Thu, Apr 21, 2016, 7:49 PM Darren Shepherd [email protected] wrote:
Every enterprise I work with wants a trusted repository of data. This is a model they are very familiar with. Think rpm/deb repo or Java's maven repos. How does this name delegation model work for this model? For example imagine nginx comes from nginx.com. But in the "enterprise" they want to mirror the images from nginx.com to their local servers such that when a user pulls "nginx" it goes to the on prem repo and not the internet.
— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/opencontainers/image-spec/issues/11#issuecomment-213218303
Does this work today already on rkt or appc? I'd like to understand more. I'm trying to wrap my head around this. The fact that it's model after go is the problem for me. Go has so many distribution problems where other languages like node and Java don't really. But the fundamental difference in the two is that node/Java have a centralized de facto store for everything (npm/mvn central). I have an easier time grasping the idea that name resolution is not decentralized but instead done based on a small set of chosen trusted sources.
@ibuildthecloud Is what you are suggesting actually, technically different than an indefinitely caching proxy with a hand written white list? In that case, such proxys already exist, and no software on OPCs side is necessary. Just set up your "in enterprise" systems to use the companies proxy, and curration and trust management can happen when writting the white list.
Is there room for TUF's delegation notions to get incorporated here? The have a pretty fine grained way of delegating trust for certain packages to different sources, with a well-studied set of security / authenticity guarantees.
On Fri, Apr 22, 2016 at 09:08:29AM -0700, Kamal Marhubi wrote:
Is there room for TUF's delegation notions to get incorporated here?
Notary [1,2] is a wrapper around TUF. More notes on that and thoughts about trust delegation here 3.
Subject: Signature verification after image-format translation
Date: Fri, 15 Apr 2016 14:50:18 -0700
Message-ID: <[email protected]>
On Fri, Apr 22, 2016 at 9:08 AM Kamal Marhubi [email protected] wrote:
Is there room for TUF http://theupdateframework.com/'s delegation notions to get incorporated here? The have a pretty fine grained way of delegating trust for certain packages to different sources, with a well-studied set of security / authenticity guarantees.
Namespaces in TUF and in particular Notary can use a DNS form.
Here is a concrete example from the docs: https://github.com/docker/notary/blob/master/docs/getting_started.md#understand-notary-naming
Here are the three use cases as I see it:
-
User doesn't care at all about name delegation, they use an existing registry for all images (e.g.
gcr.io/brendanburns/foo,gcr.io/brendanburns/bar, ...) -
User is running inside their own data center, doesn't trust anyone, doesn't want to go outside their LAN. In this case user still doesn't care about delegation, but they run their own registry and push all images their: (e.g.
my-registry.internal/foo,my-registry.internal/bar)
Neither of those two care about name delegation.
- The interesting middle use case is when someone wants to establish domain based policy (e.g. 'you can only run images from
my-company.com) but doesn't want to run their own registry. In that context, they want their end-users to pullmy-company.com/bar/foobut have that image actually be hosted onmy-company.com.gcr.io/bar/foo
it seems to me that there are two questions here:
- How do you securely get/set the mapping from
my-company.comtomy-company.com.gcr.io?
Options here appear to be DNS CNAME records, or a well-known URI.
Pros for CNAME are:
- Natural fit for lots of other examples
- No serving system requirements (can use one of N different cloud DNS providers)
Cons for CNAME:
- requires taking over an entire domain, can't use (
my-company.comrequireregistry.my-company.com) - Harder for clients to write code (DNS libraries are immature compared to HTTP)
- Security requires DNSSEC which isn't fully implemented (or at all) by some providers.
Pros for well known URI:
- Easy to poke at (curl ...)
- Security uses well known HTTPS certs, etc.
Cons for well known URI:
- Extra complexity: have to run a server (though you probably already do)
- More opaque, less elegant approach.
- Can you manipulate the rest of the image path? If so how?
I think the answer here should be "no" the risks are just to big and the win is minimal.
@brendandburns To clarify, are you saying that the user should be able to configure the naming authority for my-company.com? If a user pulls, my-company.com/bar/foo and gets my-company.com.gcr.io/bar/foo, what is the name of the resulting image?
- Can you manipulate the rest of the image path? If so how?
I think the answer here should be "no" the risks are just to big and the win is minimal.
We can agree, here. Allowing remotes or local to manipulate paths quickly gets out of hand.
Put differently, if we expressed naming as a function f(name) -> y, what is the type of y? Can we express the naming problem in this manner?
@stevvooe the name of the image is still my-company.com/bar/foo just like if you visit www.company.com and it CNAME's to storage.googleapis.com your browser still shows www.my-company.com in the URL bar.
@stevvooe put another way, this all happens in the DNS subsystem, so the client code pulling the image doesn't even notice. They just see my-company.com resolve to 1.2.3.4 which happens to be the IP for my-company.com.gcr.io
On Wed, Apr 27, 2016 at 3:27 PM Brendan Burns [email protected] wrote:
The interesting middle use case is when someone wants to establish domain based policy (e.g. 'you can only run images from my-company.com) but doesn't want to run their own registry. In that context, they want their end-users to pull my-company.com/bar/foo but have that image actually be hosted on my-company.com.gcr.io/bar/foo
Yes, exactly.
it seems to me that there are two questions here:
- How do you securely get/set the mapping from my-company.com to gcr.io?
Options here appear to be DNS CNAME records, or a well-known URI
CNAME doesn't work if you want TLS delegation to be "secure", right?
I think of delegation based on DNS/well-known URI as a "globally distributed default". If you want to override the default you need to configure that policy at a cluster/machine level, similar to download credentials.
Depending on the language we come up with here it will look like something like this:
"delegationOverrides": [
{
"name": "example.com/*",
"mirrors": ["gcr.io"]
},
{
"name": "quay.io/*",
"mirrors": ["gcr.io"]
}
]
- Can you manipulate the rest of the image path? If so how?
I think the answer here should be "no" the risks are just to big and the win is minimal.
+1 the entire name should be sent to the delegating party.
It seems to me that TLS should be fine, as long as the thing you are delegating to holds the cert for the original registry.
I think there are two cases to consider (and maybe this is the source of confusion) there is the pure delegation case: "I don't have a registry, but I want a vanity domain" and the mirror case "There's some registry out there, but for various reasons, I want to actually access those images at some other mirror location".
I was really mostly thinking about the former. The mirror one is sort of interesting but doesn't seem like a server-side protocol, but rather a client-specific configuration.
It seems to me that TLS should be fine, as long as the thing you are delegating to holds the cert for the original registry.
I don't see how we can dismiss TLS here. We still need to provide secure connections to both the white label domain and the upstream. They may be different.
How can this happen at a pure dns-level if the primary and delegate domains may have different implementations and certificates? If it really is just pure DNS, why not just tell users to use a CNAME and be done with this? If it really is just CNAMEs, do we even need to specify anything?
@stevvooe I'm not dismissing TLS. The CNAME option would involve a user defining a whole subdomain (e.g. registry.mycompany.com) creating a new different cert just for that subdomain, and storing that cert with their registry provider. And then CNAME over to the registries DNS, (similar to the flow here: https://www.keycdn.com/blog/cdn-ssl-made-easy/)
I think the goal in the spec is to design the best practice for how it would work, so that implementors know what they need to build (e.g. an interface that accepts a cert), even if the actual mechanism doesn't require an implementation.
Anyway, CNAME is one of the two options. A well-known URI is the other option. I tried to list the pros and cons of each in my earlier post
I think that the best effort in this issue is to decide which option is preferred by the maintainers.
If it is easier for me to put the options and pros/cons into a markdown doc and have the discussion in a PR, I can do that too. (and if I missed any pros/cons let me know)
@brendandburns I see. Sorry about the harsh wording here.
How is this different from using TLS with a CNAME as one normally would? It sounds like a very standard usage. Am I missing something? Why does it need to specified? Seems like it belongs in recommendations or best practices.
As an interesting data point, I have suggested this approach situations looking "spruce" up the registry urls. Typically, this is easy if you host your own registry. The problem arises when using providers such as AWS ECR, Docker Registry and GCR (please correct me if I am wrong). They would have to host a private key for each domain. Would such a provision push a requirement for services running a registry to accept certificates to use for white labeling?
@stevvooe In my mind, hosting keys isn't really a challenge for most cloud providers, since they already have to do it in order to terminate SSL for their load balancers.
Regarding spec. versus best practice, I guess the point is that this issue says: "define a spec" if we decide that CNAME is what we would suggest then we'll just close this issue as "done" ;)
In my mind, hosting keys isn't really a challenge for most cloud providers, since they already have to do it in order to terminate SSL for their load balancers.
In principal, but are any registry providers currently offering this? I couldn't find it in gcr or ecr and I don't think Docker Hub has it either.
Either way, I am not sure that CNAME is an interesting case. It sounds like more interesting topic is decoupling naming from location.
Is there some technical reason why this discussion is termed "name delegation" and not "discovery?" It seems so much simpler if we talk about discovery. That is just defining a process of looking at X, Y, and Z in some specific manner.
In my understanding, name delegation and discovery are two related processes, and one is harder than the other ;). Name delegation is the science of creating unique identifiers, it typically requires either centralization or randomness. In the DNS system, name delegation is centralized, when you create a new domain name, you reserve it in a central registrar. However, DNS lookup (aka discovery) is de-centralized, you typically use the DNS server provided to you by your ISP, and indeed, there is no way to even access the central registry for most devices.
I think that both problems are interesting, though I feal name delegation is FAR FAR FAR more vital in terms of "lets not f--- this up", as it is much harder to change and fix after the standard is published.
On 04/29/16 04:19, Darren Shepherd wrote:
Is there some technical reason why this discussion is termed "name delegation" and not "discovery?" It seems so much simpler if we talk about discovery. That is just defining a process of looking at X, Y, and Z in some specific manner.
— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/opencontainers/image-spec/issues/11#issuecomment-215612525
@stevvooe CNAME is one way of decoupling naming from location. It's a way of saying: "for this location, the canonical (that's the 'C') location is this other location over there"
and though no one currently provides it, it seems that the point of developing a specification is to encourage providers to add support.
Again, in my mind there are two options with the pros and cons outlined above. Though personally I prefer the CNAME solution, it seems like the purpose of this issue is to get to a "(dis)agree and commit"
I'm not sure what the process is for that. Shall we just have a vote?
@brendandburns Yes, the purpose is to get to a solution and commit. Ping the other maintainers: @jonboulle @jstarks @jbouzane @vbatts to chime in on how they feel. If you are coming new to this thread see @brendandburns excellent write-up on pros/cons of various options for name delegation.
Also, paging @josephschorr from the Quay.io team as well as we have considered this feature for quite some time (we called it "white label").
Personally, I am leaning towards well-known as it gives a "zero trust" sort of model to delegation. It is also is used by other "modern" internet plumbing stuff like OpenID Connect.
@brendandburns, @stevvooe: Speaking just from the perspective of the registry, as @philips alluded to, we've considered the idea of "white labeling" via CNAME for some time. The original goal was to allow name owners (e.g. "redhat.com") to delegate pulls from their own domain to Quay, such that docker pull cooldomain.com/someimage would be rewritten as docker pull quay.io/someorgforcooldomain/someimage. We quickly realized that there were two major issues with doing so:
- It wouldn't really work for the top-level domains when used for the internet as a whole, which means that a subdomain would have to be used (e.g. "docker.cooldomain.com")
- We'd have to store private keys as all our traffic is SSL terminated.
The SSL issue in particular is the reason we did not move forward, as storing private keys on behalf of a variety of users is a concerning proposition. Well-known has the nice benefits of working with top-level domains and not requiring us to store keys, so we'd prefer that route from our perspective. It also has the nice added bonus of allowing domain owners to choose dynamically to serve different images from different registries.
