vsphere-graphite icon indicating copy to clipboard operation
vsphere-graphite copied to clipboard

backend elastic is not compatible with v7

Open hamelg opened this issue 6 years ago • 3 comments

2019/06/05 21:09:59 Version information: NA - NA@NA (NA)
2019/06/05 21:09:59 Starting daemon: vsphere-graphite
2019/06/05 21:09:59 main: requested properties memorysizemb, host, tags, cluster, network, resourcepool, folder, numcpu, disks, datastore, urls
2019/06/05 21:09:59 vcenter 127: initializing
2019/06/05 21:09:59 vcenter 127: connecting
2019/06/05 21:09:59 vcenter 127: disconnecting
2019/06/05 21:09:59 backend elastic: initializing
2019/06/05 21:09:59 backend elastic: 127.0.0.1:9200/vcsim-2019.06.05
2019/06/05 21:09:59 elastic: error creating elastic index vcsim-2019.06.05 - elastic: Error 400 (Bad Request): Failed to parse mapping [_doc]: Root mapping definition has unsupported parameters:  [doc : {properties={Timestamp={format=epoch_second, type=date}}}] [type=mapper_parsing_exception]
2019/06/05 21:09:59 Could not initialize backend Error:  elastic: Error 400 (Bad Request): Failed to parse mapping [_doc]: Root mapping definition has unsupported parameters:  [doc : {properties={Timestamp={format=epoch_second, type=date}}}] [type=mapper_parsing_exception]

With Elastic search 7.0 , types were removed and when creating a mapping it no longer accepts types which is a breaking change.

Reference Urls: https://www.elastic.co/guide/en/elasticsearch/reference/master/removal-of-types.html

hamelg avatar Jun 05 '19 19:06 hamelg

I need time to check this. To solve last bug I had to force v5 library .

I must admit this was because I had no time to go into go modules required to use the v7 library.

cblomart avatar Jun 06 '19 04:06 cblomart

@MnrGreg did the initial development on this.

I had the discussion on a recent Elastic training about how to handle elastic on an application-level (@melvynator).

I feel that rather than upgrade to the elastic v7 library, we should foresee a simpler backend whos role would only be to push data to elastic. In this sense index management would be by default left to the elastic administrator (mapping or mapping template, ilm, ...).

I would propose the following:

  • simple post to bulk API
  • create a mapping template (optional; config file)
  • use a template for index (i.e vsphere-yyyy-mm-dd; optional; config file)

Would you implement that as an alternative to the elastic backend or as a replacement for it?

cblomart avatar Mar 22 '20 12:03 cblomart

+1 for a simple backend. If you need some testing, I'll be happy to help you.

hamelg avatar Mar 23 '20 07:03 hamelg