[BUG] Scrape failure logs with info/debug level

[patmaddox]

(just saving this for later, will fill in more info)

Not quite sure what's going on. It's prefixed with [info] but claims level=debug. This is for an error that we weren't able to diagnose until we reduced the logger level.

[info] Grafana Agent log: "ts=2022-07-27T20:20:42.78545Z caller=scrape.go:1302 level=debug agent=prometheus
instance=7456140ad35b2bc58fd30b43a2b07375 component=\"scrape manager\" scrape_pool=sprlcl
target=https://localhost:4000/metrics msg=\"Scrape failed\"
err=\"Get \\\"https://localhost:4000/metrics\\\": http: server gave HTTP response to HTTPS client\""

[patmaddox]

I suspect maybe there’s little/nothing to do here? Because the agent is running in its own OS process, it doesn’t actually share the elixir logger.

So at best we could configure where agent logs go, and treat them separately from the elixir app?

[akoutmos]

I have had issues with Grafana Agent log levels in the past. I.e when I set it to a certain level, I still get lower level logs. Perhaps this has been fixed in later versions? I'll make a PR to update the available GA versions that PromEx supports.