Create jsonnet package for the docker plugin

[donny-dont]

I think it would be useful, especially if the drone-agent can integrate with the jsonnet package manager, for us to provide a plugin.libsonnet for each of the drone-plugins repositories as a way to make it easier to correctly use the plugins.

Here's a first stab at things and I'd like feedback on what things should look like.

An example usage is

local docker = import 'templates.libsonnet';
local settings = 
  docker.Settings('foo/bar') +
  docker.AutoTag(1809) +
  docker.Squash +
  docker.DisablePull;

[
  docker.WindowsPlugin('foo', settings),
]

This outputs

[
  {
    "image": "plugins/docker",
    "name": "foo",
    "settings": {
      "auto_tag": true,
      "auto_tag_suffix": 1809,
      "context": "",
      "daemon_off": true,
      "dockerfile": "Dockerfile",
      "pull_image": "false",
      "purge": "false",
      "repo": "foo/bar",
      "squash": true
    },
    "volumes": [
      {
        "name": "docker_pipe",
        "path": "\\\\\\\\.\\\\pipe\\\\docker_engine"
      }
    ]
  }
]

@metalmatze @tboerger

[metalmatze]

I don't think that the drone-agent should be concerned with jsonnet at all, to be quite honest. For me this is part of writing .drone.yml, if it is generated or not, doesn't really matter. So far, what I got from the conversation on chats, was that you would like to have something like a stdlib for drone pipelines/steps?

[donny-dont]

@metalmatze in this case my thought is that each drone-plugin repo should have an associated plugin.libsonnet that you could import using jsonnet-bundler. This would let users easily get a plugin up and working.

Some goals for that are

• The libsonnet files should all be consistent in how you use them.
• They should promote best practices

The docker plugin is a good candidate to start playing with things since there are a large amount of options. It also has some interesting things to worry about like Windows support.

I'll do a review on this code to get more in depth here. Really appreciate you taking a look since I'm relatively new with jsonnet.

[metalmatze]

Ok. Seems like I misunderstood. I agree then. Yes, that's exactly the intended way. Have a repository that has the master libsonnet and import that with jsonnet-bundler to every repository.

I've just started working a bit on a jsonnet pipeline, that I would write for this docker plugin.

[metalmatze]

Here's the first experiment I did writing such a drone.libsonnet. Especially the first pipeline called testing, is exactly like the current one in this repo. A few things missing, didn't get to them today.

https://github.com/drone-plugins/drone-docker/commit/bfd085312f9001d60d8bb3b3ef8fc1f65ea34bfd

Note it only merges, no functions and no ifs.

[bradrydzewski]

So if I understand this correctly, the goal would be for every plugin in the organization to have a plugins.jsonnet file that can be imported as a library.

For example:

github.com/drone-plugins/drone-docker/tree/master/jsonnet/plugin.libsonnet github.com/drone-plugins/drone-slack/tree/master/jsonnet/plugin.libsonnet github.com/drone-plugins/drone-s3/tree/master/jsonnet/plugin.libsonnet

Then I could use the jsonnet bundler to import these libraries to use them in my project. The goal for each library would be to provide plugin-specific operations (functions, mixins, etc) to reduce boilerplate, provide sane defaults, and prevent common user configuration errors.

@metalmatze if I understand correctly this has similar goals to the drone.libsonnet library you were working on, but focuses solely on plugins, and takes a distributed approach (each plugin has its own library) vs a central approach (single repository with many libraries).

[donny-dont]

@bradrydzewski yep! That's the idea here

[metalmatze]

Ah, I didn't get that there are 2 problems to solve. Both make sense. For the plugins it's mostly about the step definition, is it?! Then I propose to only expose that step definition so that one can do something like:

local drone = (import 'drone/drone.libsonnet');
local docker = (import 'drone-docker/plugin.libsonnet');

...
{
  steps: [
    docker {
      name: 'docker-quay',
      repository: 'quay.io/metalmatze/foobar',
    },
  ],
},

Note: This is missing the pipeline definition.

[donny-dont]

@metalmatze how does the above convey that you need to have the repository value in there to even make this work?

[metalmatze]

Worked the last hour on a small example. Still has rough edges (again like yesterday), but should get the main ideas across.

plugin.libsonnet:

{
  name: 'docker',
  image: 'plugins/docker',
  repo:: 'foo/bar',

  settings: {
    repo: $.repo,
  },

  // Of course there can be other registries.
  quay+:: self + {
    name: 'docker-quay',

    settings+: $.settings {
      registry: 'quay.io',
      repo: 'quay.io/%s' % $.repo,
    },
  },
}

Note: The file above would be downloaded by jsonnet-bundler and drone jsonnet needs to support vendor/ folders.

.drone.jsonnet:

local docker = (import 'plugin.libsonnet');

{
  platform: {
    os: 'linux',
    arch: 'amd64',
  },

  steps: [
    // The bare minimum with defaults copied from the docs
    docker,

    // Changing the name and repository.
    // Note how repo is top level but printed inside of settings later
    docker {
      name: 'foobar',
      repo: 'foo/bar',
    },

    // Setting the repo directly inside of settings is still possible
    docker {
      settings+: {
        repo: 'something/else',
      },
    },

    // Now we have a preset for quay as registry.
    docker.quay {
      name: 'docker-quay',
      repo: 'cool/repository',
    },
  ],
}

Result .drone.yml:

---

platform:
  os: linux
  arch: amd64

steps:
- name: docker
  image: plugins/docker
  settings:
    repo: foo/bar

- name: foobar
  image: plugins/docker
  settings:
    repo: foo/bar

- name: docker
  image: plugins/docker
  settings:
    repo: something/else

- name: docker-quay
  image: plugins/docker
  settings:
    registry: quay.io
    repo: quay.io/foo/bar

...

For me this would all be about the fundamental building blocks that one might need to a plugin. I wouldn't even support that much more. Benefits would be to have updated versions etc for new plugins, by simply running jb update.

Everyone is still free to build on top of those fundamental and create the abstractions they need for their organization. Thus all developers of that organization would only indirectly use this plugin.libsonnet. Let me know if I should elaborate on this last part.

[donny-dont]

Ok I'm following things better now @metalmatze but I'm still wondering about a couple things.

So I think there's an error thing you can use so if you don't provide a repo then you can just throw an error. This seems to be fine to me.

What about things like authentication? Or auto-tag? I have those as functions. How would that work here with your setup?

[metalmatze]

Thanks for taking a look @sparkprime! Looking at this from a higher level view. In what way would you structure these plugins? When would you use the composition and when functions? What do you think about the example I made above?

[sparkprime]

Your example looks fine to me. It's a bit weird to have the docker object be a step as well as having other templates in it, but it does work. Alternatively you could have docker_steps.docker and docker_steps.quay. Unrelatedly, I would do this:

    settings+: {  // you have the + already, so don't need $.settings
      registry: 'quay.io',
      repo: 'quay.io/%s' % super.repo,
    },

[sparkprime]

How about this:

{

  base:: {
    local plugin = self,
    name: 'docker',
    image: 'plugins/docker',
    repo:: 'foo/bar',
    settings: {
      repo: plugin.repo,
    },
  }

  // Of course there can be other registries.
  quay:: self.base + {
    name: 'docker-quay',
    settings+: {
      registry: 'quay.io',
      repo: 'quay.io/%s' % super.repo,
    },
  },
}

If you then do docker.quay { repo: "foo" }, you'll get settings.repo == "quay.io/foo" like in your original code.

[metalmatze]

Really cool! That looks even better from my point of view. For a normal docker step one would do docker.base{ repo: "foo" }, right?

[sparkprime]

Yeah