Cortextool loki rules load error tls: failed to verify certificate: x509: certificate signed by unknown authority

[EstefaSalazar]

Hi everyone,

I'm trying use cortex-tool to interact with my grafana loki for upload rules to S3. In the command "cortextool rules load", I use in the --address flag the endpoint of loki-distributed-ruler and when I do it manual it works but I need to do it inside a pod in the comand/arg fields and that's when the error occurs:

group: 'EKS linea base logs', ns: '03_alert_ruler_logs_projname'
/api/prom/rules/03_alert_ruler_logs_projname/EKS%20linea%20base%20logs
time="2023-11-25T16:19:36Z" level=error msg="error during request to cortex api" error="Get \"https://internal-endpointEKS/loki/api/prom/rules/03_alert_ruler_logs_projname/EKS%20linea%20base%20logs\": tls: failed to verify certificate: x509: certificate signed by unknown authority" method=GET url="https://internal-endpointEKS/loki/api/prom/rules/03_alert_ruler_logs_projname/EKS%20linea%20base%20logs"
cortextool: error: load operation unsuccessful, unable to contact cortex api: Get "https://internal-endpointEKS/loki/api/prom/rules/03_alert_ruler_logs_projname/EKS%20linea%20base%20logs": tls: failed to verify certificate: x509: certificate signed by unknown authority, try --help

I tried to use the flags --tls-key-path and --tls-cert-path with a trust certificate but the result is the same.

Is there a way to tell cortex-tool to ignore tls verification or another way to indicate trust certificates?

Thanks!!

[m4vr0x]

Hi @EstefaSalazar, doing some necroposting here but just in case it's useful to someone....

I had the same issue with the Grafana command (cortextool analyse grafana) and I was able to make it work using SSL_CERT_FILE environment variable:

export SSL_CERT_FILE=/path/to/certificates/internet_certs_bundle.pem

I tested it with success in Alpine container and also on macOS (despite the mention in the doc)

See the reference from Go documentation: https://pkg.go.dev/crypto/x509#SystemCertPool Since it's a go env var, I suspect it would work with cortextool rules load command also.

And also specifically for the rules command, I found this in the code:

		c.Flag("tls-ca-path", "TLS CA certificate to verify cortex API as part of mTLS, alternatively set CORTEX_TLS_CA_PATH.").
			Default("").
			Envar("CORTEX_TLS_CA_CERT").
			StringVar(&r.ClientConfig.TLS.CAPath)

I assume CORTEX_TLS_CA_CERT could also work for you