Move client side image pull logic to engine-api

[ibuildthecloud]

Honestly, it's rocket science to pull an image. Ignore the black magic required to do trusted pulls, just parsing a string to ImageID and Tag is not even straight forward. Let alone who knows what RegistryAuth is supposed to be. It would be greatly appreciated to move as much logic from https://github.com/docker/docker/blob/master/api/client/pull.go to this project in the form of some util methods or something.

I see that as a particular problem when it comes to signed/trusted images. While Docker may have all this great signing stuff, if clients outside of the docker CLI can't figure out how to use it, then that's a problem. Honestly I don't even know what I'm talking about here because I don't even fully understand what is required to do trusted pulls. I just look at https://github.com/docker/docker/blob/master/api/client/trust.go and get overwhelmed.

[thaJeztah]

/cc @docker/core-notary-maintainers

[endophage]

@ibuildthecloud thank you for the feedback. I'll try to provide some explanation on how trusted pulls work specifically, then let you decide if you still want changes made to the APIs.

First, trusted pulls rely on a previously existing feature, "pull-by-digest". In this flow, engine is provided with the image name and a checksum of the desired version (in place of the normal tag). This is secure vs a typical pull by tag because the user is specifying the exact content they want to retrieve, rather than a name which could be associated with any content (in a pull by tag it's up to the server to resolve the name).

Content trust then leverages pull-by-digest and moves the name resolution from the server, to the docker CLI. The docker CLI retrieves and validates the trust data for the image repository, looks up the provided tag in that trust data, and receives the checksum (sha256) of the image associated with that tag. It is then able to execute a secure pull-by-digest against the engine.

[stevvooe]

@ibuildthecloud Ignore the black magic required to do trusted pulls, just parsing a string to ImageID and Tag is not even straight forward.

I noticed this yesterday while looking into the API client library a little more. I filed https://github.com/docker/engine-api/issues/137 in response. If you dig down into the example, it basically shows us just giving a ref to the method and have the engine-api sort out the parsing.

This was actually the goal of the reference package, but that effort needs a little more consolidation.

[calavera]

I'd rather not do this tbh. This package is designed, as much as possible, to only keep logic shared between API client and server. I agree with @stevvooe that we should work on consolidating the reference package.

[bfirsh]

@ibuildthecloud I agree that more client logic should live in a reusable library somewhere. Another example is simply running a container, which is also rather convoluted with this library.

@calavera If this package is designed to keep logic shared between API client and server, perhaps we should create another project as the user-facing library for building apps on top of the Docker API? Something akin to docker-py or dockerode, but for Go. WDYT?

[vdemeester]

I think it could in some engine-cli project used by docker/docker and designed in a reusable way 😇

[stevvooe]

@vdemeester It may also make sense in distribution.