#257: Add gauge for total fields mapped in an index

[jnadler]

This PR replaces #364 so that I can take over the work started by @msodi. The original PR feedback from @sysadmind has been addressed.

[jnadler]

@sysadmind @zwopir anyone willing to review this change? I think a lot of folks who use ES for logging will appreciate this change, logging often causes field cardinality explosions and being able to alert on those before they cause an outage is a big win. Thanks!

[jnadler]

Thanks for the feedback @sysadmind !! Just pushed some updates.

[jnadler]

Just tested this against one of our prod clusters, the new gauge elasticsearch_indices_mappings_stats_field_count is working as expected.

[jnadler]

Thanks @sysadmind, appreciate your help! @zwopir would you please merge?

[repl-mike-roest]

Any motion on this PR? This is something we would really like to have as well

[jnadler]

@repl-mike-roest no PRs have been merged in 15 months; we're running a build off this branch since september with good results. @zwopir any chance you'd be willing to add others (maybe @sysadmind ?) as admins on this project so that they can merge and release to keep it alive?

[repl-mike-roest]

@jnadler I've been trying to build it locally as well to test if it will work with what I'm trying to pull from our clusters. However I seem to be hitting a build failure when trying to build the docker container

docker build -t elasticsearch_exporter .
<docker output>
go: cannot find main module, but found vendor/vendor.json in /go/src/github.com/justwatchcom/elasticsearch_exporter
        to create a module there, run:
        go mod init
>> formatting code
go: cannot find main module, but found vendor/vendor.json in /go/src/github.com/justwatchcom/elasticsearch_exporter
        to create a module there, run:
        go mod init
make: *** [Makefile:37: format] Error 1

I tried updating the docker file to use a go 1.13,1.14 and 1.15 base image from the quay.io/prometheus/golang-builder and all of them fail with slightly different errors (errors above are with the Dockerfile as is so using latest which seems to be 1.16)

Any ideas?

[jnadler]

try go mod vendor then make docker

[repl-mike-roest]

I looks like something that is broken upstream with promu https://github.com/prometheus/promu/issues/218 if I update the docker file back to quay.io/prometheus/golang-builder:1.15-main then I get the same error that is reported in the promu issue.

[jnadler]

strange i got a different vendoring error until running go mod vendor but was fine with either make build or make docker after that; i presume there's been some drift in the dependencies

i'm on go 1.16.2 but not sure it should matter; make docker does the actual go build inside a container so it should be consistent for everyone i'd think, that should prevent most classes of "builds on my machine" issues

[repl-mike-roest]

With docker 1.16.2 if I do

go mod init  (this is needed cause if I just do go mod vendor it just complains there is no go.mod file)
go mod tidy
go mod vendor
make build

I get a error go: inconsistent vendoring in /Users/mike.roest/Workspaces/elasticsearch_exporter:

Doing another go mod vendor then make build again all the package issues seem to be resolved but getting an actual compile failure:

/main.go:170:27: undefined: prometheus.Handler
!! command failed: build -o /Users/mike.roest/Workspaces/elasticsearch_exporter/elasticsearch_exporter -ldflags -s -X github.com/justwatchcom/elasticsearch_exporter/vendor/github.com/prometheus/common/version.Version=1.1.0 -X github.com/justwatchcom/elasticsearch_exporter/vendor/github.com/prometheus/common/version.Revision=ab8add4ff4188654f4af7b1a1498679080de80e8 -X github.com/justwatchcom/elasticsearch_exporter/vendor/github.com/prometheus/common/version.Branch=HEAD -X github.com/justwatchcom/elasticsearch_exporter/vendor/github.com/prometheus/common/version.BuildUser=mike.roest@cgdv-b0105 -X github.com/justwatchcom/elasticsearch_exporter/vendor/github.com/prometheus/common/version.BuildDate=20210416-11:54:19  -a -tags netgo github.com/justwatchcom/elasticsearch_exporter: exit status 2
make: *** [build] Error 1

Then following this https://github.com/freenetdigital/prometheus_oracle_exporter/issues/22 suggestion I now have it building with make build and make docker

[repl-mike-roest]

I've expanded on your changes here @jnadler to also add the total_fields limit setting https://github.com/repl-mike-roest/elasticsearch_exporter/compare/55deb7329581abd06007a591e74fe25a4d7c2b34...9e9be05960db3cc90eb84ae42ba0c8dfa6fd227c

This allows for adding alerts on a total_fields approaching the limit

[jnadler]

that's great @repl-mike-roest ! Hey @zwopir totally understand that you're busy but would you consider making someone else an admin? Or even turning this repo over to someone else who will review/merge PRs and do releases?

[jnadler]

Hey @SuperQ ... I see that you've been maybe merging things and even did a release recently, it's so good to see that someone is keeping this thing alive!

Any chance of merging this?

[sysadmind]

This needs a rebase before it can be merged. After a rebase, I can take a look just to make sure everything still looks good and merge. Thank you!

[jnadler]

Equivalent functionality appears to have been merged in https://github.com/prometheus-community/elasticsearch_exporter/pull/411