jaeger icon indicating copy to clipboard operation
jaeger copied to clipboard

[Bug]: es.index-prefix not consistent in es jobs

Open OpenGuidou opened this issue 3 years ago • 5 comments

What happened?

To be able to configure the index-prefix with ES, the flag is es.index-prefix for the collector and query components. For the jobs (es-index-cleaner, es-rollover and es-mapping-generator), it's using directly index-prefix without the prefix.

If we're using a common configuration for all components, then the jobs are not configured correctly with the indexprefix. Example with the jaeger helm chart:

https://github.com/jaegertracing/helm-charts/blob/main/charts/jaeger/templates/_helpers.tpl#L371

Steps to reproduce

Deploy the jaeger helm chart with the elasticsearch storage enabled. The jobs won't be able to find the indexes.

Expected behavior

It should be aligned and use the es. prefix as well

Relevant log output

No response

Screenshot

No response

Additional context

No response

Jaeger backend version

Latest

SDK

No response

Pipeline

No response

Stogage backend

Elasticsearch

Operating system

any

Deployment model

Any

Deployment configs

No response

OpenGuidou avatar Mar 01 '23 14:03 OpenGuidou

If the helm chart is using incorrect flags, we should fix the chart.

yurishkuro avatar Mar 01 '23 15:03 yurishkuro

Then why use the prefix for the username and password ? https://github.com/jaegertracing/jaeger/blob/main/cmd/es-index-cleaner/app/flags.go#L29

Was it not to align the flags across all usages ?

OpenGuidou avatar Mar 01 '23 15:03 OpenGuidou

It's fine to align all usages, but the way we do it is by supporting both new and old flags for some time, see https://github.com/jaegertracing/jaeger#version-compatibility-guarantees

yurishkuro avatar Mar 01 '23 15:03 yurishkuro

Fair enough, thanks I'll update de PR based on that

OpenGuidou avatar Mar 01 '23 16:03 OpenGuidou

I ran into this bug this week. I had to go through the es-index-cleaner code to understand why my indices where not cleaned as per requested (No indices to delete was the only indication in the logs of the container although it was listing all indices). No indication of any sort regarding the indexPrefix being incorrect...

I had to understand how env vars were used in the es-index-cleaner to notice it was not working with the ES_INDEX_PREFIX envvar name passed by default but was expecting INDEX_PREFIX instead...

If you don't want to solve this, at least update the documentation to mention that INDEX_PREFIX in esIndexCleaner.extraEnv should be set when using storage.elasticsearch.indexPrefix ...

rbaret avatar Feb 16 '24 10:02 rbaret

we will address this in jaeger-v2, where we're redesigning configuration completely.

yurishkuro avatar Jun 08 '24 19:06 yurishkuro