Hiera not merging output options correctly.

[Cova]

I use the graphite output of telegraf, and all configuration is done via Hiera. In my common.yaml I have defined:

telegraf::outputs:
  graphite:
    - servers:
      - "graphite:2003"
      template: "host.tags.measurement.field"

And in various other files throughout the hierarchy I also add an extra option to the output using:

telegraf::outputs:
  graphite:
    - prefix: server-role

In order for this to merge correctly, I need to use Hiera's deep-merge with the 'merge_hash_arrays' option enabled - but even after enabling it using the lookup_options in my common.yaml file it is still not merging correctly. The 'lookup' calls in init.pp both force "merge = deep" which overrides any settings I make.

I'm not sure which is the better solution - but init.pp needs to be changed to either not specify a merge behavior so that the options defined in hiera lookup_options hash take effect, or to specify {'strategy' => 'deep', 'merge_hash_arrays' => true} The first method allows users to override the behavior if the choose, but is probably broken by default if they don't have lookup_options defined - the second will work by default, but forces it on everyone.

[htj]

+1 for having this use case. We often have hosts that need to collect some service specific metrics, and adding those is a bit cumbersome.

We ended up doing this in inputs in the resource spec:

    inputs              => {
      ...
      mem    => [ {} ],
      io     => [ {} ],
      net    => [ {} ],
      system => [ {} ]
    } + lookup('ndn::telegraf.inputs', {default_value => [{}], merge => deep}),

So we could add additional inputs per host.

[alexjfisher]

@htj The ability to configure merge behaviour was added in https://github.com/voxpupuli/puppet-telegraf/pull/134 Does this work for you?

[htj]

@alexjfisher

I had several issues getting this to work. Having: merge_hash_arrays: true Option in common.yaml as listen in: https://github.com/voxpupuli/puppet-telegraf/pull/99/commits Make it possible to have multiple inputs of the same type (we often have multiple inputs.exec for custom collections)

However this did not make it possible to have multiple telegraf::inputs: As one would always wine. I suspect it has something to do with look hiarchy, but I don't understand Puppet well enough to debug it.

[alexjfisher]

The lookup_options in https://github.com/voxpupuli/puppet-telegraf/blob/777e608c000655781179913d9ee41aacd216c8c0/data/defaults.yaml are just the defaults. You can override lookup_options in your environments hiera, for example in your node specific hiera data. See https://puppet.com/docs/puppet/latest/hiera_merging.html#overriding-merge-behavior

If you still can't make it merge the way you need it to, could you provide some specific examples of what you're trying to achieve?

[danielsreichenbach]

Changing lookup_options in the module to this:

---
lookup_options:
  'telegraf::inputs':
    merge:
      strategy: deep
      merge_hash_arrays: true
  'telegraf::outputs':
    merge:
      strategy: deep
      merge_hash_arrays: true

resolves the issue.