Butane with multiple merge.local does not convert to ignition

[ddt3]

I am using the following Butane file that I cannot get to work:

variant: fcos
version: 1.5.0
ignition:
  config:
    merge:
    - local: ign/include/ssh.ign
    - local: ign/include/add-compose.ign
    - local: ign/include/music.ign
    - local: ign/include/start-docker.ign
storage:
  files:
  - path: /etc/hostname
    contents:
      inline: examplehost

This is the output:

$ butane yaml/example-local.yaml -d /home/dries/workspaces/proxmox-coreos 
error at $.ignition.config.merge.1, line 8 col 7: duplicate entry defined
error at $.ignition.config.merge.2, line 9 col 7: duplicate entry defined
error at $.ignition.config.merge.3, line 10 col 7: duplicate entry defined
Error translating config: config generated was invalid

When I use source: it just works

variant: fcos
version: 1.5.0
ignition:
  config:
    merge:
    - source: http://www-fc.familie-dokter.lan/ign/include/ssh.ign
    - source: http://www-fc.familie-dokter.lan/ign/include/add-compose.ign
    - source: http://www-fc.familie-dokter.lan/ign/include/music.ign
    - source: http://www-fc.familie-dokter.lan/ign/include/start-docker.ign
storage:
  files:
  - path: /etc/hostname
    contents:
      inline: examplehost

This file converts without a problem to ignition:

$ butane yaml/example.yaml
{"ignition":{"config":{"merge":[{"source":"http://www-fc.familie-dokter.lan/ign/include/ssh.ign"},{"source":"http://www-fc.familie-dokter.lan/ign/include/add-compose.ign"},{"source":"http://www-fc.familie-dokter.lan/ign/include/music.ign"},{"source":"http://www-fc.familie-dokter.lan/ign/include/start-docker.ign"}]},"version":"3.4.0"},"storage":{"files":[{"path":"/etc/hostname","contents":{"compression":"","source":"data:,examplehost"}}]}}

Specification at: https://coreos.github.io/butane/config-fcos-v1_6/ does not explain why the behavior is different.

[prestist]

@ddt3 hello :)

So with the 'local' field we added sugar for enabling the embedding of a file into the ignition config at butane time. This is purely sugar and is only really able to be a singleton in terms of the merge section.

Usually merge is used for cloud providers, as custom-data usually has limits and a way to get around those length limits we can add a source, which ignition can fetch at ignition time. What is your use-case with mulitple embedded ignition configs ?

[ddt3]

@prestist Thank you for looking into this :-)

I think the use case can actually be derived from my example. I use flatcar on my Proxmox server.

I have a number of "standard" additions that I use to fire-up a flatcar vm

• File shares that are needed in some vms
• SSH key that is needed for all vms
• adding docker-compose binaries to be able to use docker-compose
• etc

All of this is scripted and automated: Deploying a new flatcar vm means, that I create a butane file by shopping from the "standard additions" and adding specifics for that vm. As an example: I add a specific docker-compose.yaml which is possible as I have also included the "standard addition" to install docker-compose binaries, etc. ) .

All of this is possible at the moment, but only if I place the resulting ignition files on a webserver that is reachable from my Proxmox cluster.

On Proxmox a " complete" ignition file including everything could also be stored in a Snippets storage, which means I no longer depend on an additional (web) server to deploy these vms.

Note that in addition: Promox will only allow moving a VM to another host in the cluster if the configured ignition file is available to the other Proxmox host. This is true for the Snippets Storage , these are available to all hosts in a cluster.

[prestist]

Hey! no problem :)

Ok, so these files really are constants then. And you want to layer them into each of your configs, kinda like legos. Let me look into how the sugar works, it sounds like a nice use case I will try and bring it up today in FCOS meeting!

[ddt3]

@prestist anything I can help with?

[max-zelinski]

@prestist do you have any updates by chance?

[prestist]

@ddt3 @max-zelinski

I want to bring attention to https://github.com/coreos/butane/issues/118 This is basically what you want but with the added benefit of using .bu files rather then .ign. Currently there is an on-going effort to add this.

There is work in progress, I will update this once that work lands.