Simon Pasquier

@jmcarp can you share the config that triggered the error?

Validating fields supporting inline templates when loading the Alertmanager configuration would definitely be a good addition.

I wonder if the validation could be performed in each notifier's New() since it gets the template engine as a parameter? And `amtool check-config` could replicate/mimic the [buildReceiverIntegrations()](https://github.com/prometheus/alertmanager/blob/6cbe2eb21f7cd770d8f247c545941f3a8c97f6a0/cmd/alertmanager/main.go#L133-L135) function.

> adding another dependency would make running tests without Prow more difficult #160 You can use a CircleCI account token instead of a project token (in fact this is what...

cc @SuperQ which might be interested by the discussion too.

Using the artifacts from the upstream CI, we don't have to think about how to build the binary. That being said, I understand your concerns so maybe I can add...

> It might happen that a user starts benchmarking after a long time -> when the artifacts are no longer there in circle-ci. The information I've found when looking at...

Even if not for prombench, it can be useful sometime to have the PR image available for people to test (this has been brought up in https://github.com/prometheus/prometheus/pull/4453 too). Again taking...

I've dug further and it isn't feasible with CircleCI to push a container image from a forked PR (neither with Travis btw). This boils down to security concerns: to push...

The CI job triggered after https://github.com/prometheus/prometheus/pull/6326 got merged hasn't completed though it's been started more than 30 hours ago.