alertmanager icon indicating copy to clipboard operation
alertmanager copied to clipboard
Published 3 weeks ago • verified publisher icon prometheus

TestRetry fails on Darwin

Open ryanwuer opened this issue 4 years ago • 3 comments

What did you do?

  • clone the repo
  • checkout to branch 'release-0.21'
  • first execute 'make build' , then execute 'make test'

What did you expect to see? All tests can be passed

What did you see instead? Under which circumstances? --- FAIL: TestRetry (6.51s) acceptance.go:179: stdout: acceptance.go:180: stderr: level=info ts=2021-01-21T09:39:04.729Z caller=main.go:220 msg="Starting Alertmanager" version="(version=0.21.0, branch=develop, revision=94386bf72204899a96b83e1b6d22fce1c1306e4f)" level=info ts=2021-01-21T09:39:04.729Z caller=main.go:221 build_context="(go=go1.15.3, [email protected], date=20210121-09:04:44)" level=debug ts=2021-01-21T09:39:04.729Z caller=cluster.go:149 component=cluster msg="resolved peers to following addresses" peers=127.0.0.1:50007 level=debug ts=2021-01-21T09:39:04.729Z caller=delegate.go:230 component=cluster received=NotifyJoin node=01EWJ4ZJCSG0953MEZDW06QXPM addr=127.0.0.1:50007 level=debug ts=2021-01-21T09:39:04.730Z caller=cluster.go:306 component=cluster memberlist="2021/01/21 17:39:04 [DEBUG] memberlist: Initiating push/pull sync with: 127.0.0.1:50007\n" level=debug ts=2021-01-21T09:39:04.730Z caller=cluster.go:306 component=cluster memberlist="2021/01/21 17:39:04 [DEBUG] memberlist: Stream connection from=127.0.0.1:50022\n" level=debug ts=2021-01-21T09:39:04.730Z caller=cluster.go:233 component=cluster msg="joined cluster" peers=1 level=info ts=2021-01-21T09:39:04.730Z caller=cluster.go:623 component=cluster msg="Waiting for gossip to settle..." interval=2s level=info ts=2021-01-21T09:39:04.730Z caller=cluster.go:632 component=cluster msg="gossip not settled but continuing anyway" polls=0 elapsed=1.56µs level=debug ts=2021-01-21T09:39:04.749Z caller=main.go:359 externalURL=http://MacBook-Pro-WRJ.local:50004 level=info ts=2021-01-21T09:39:04.749Z caller=coordinator.go:119 component=configuration msg="Loading configuration file" file=/var/folders/_n/mbr_vhxs7k34wmqjsrcr2fdh0000gp/T/am_test077575705/config.yml level=info ts=2021-01-21T09:39:04.749Z caller=coordinator.go:131 component=configuration msg="Completed loading of configuration file" file=/var/folders/_n/mbr_vhxs7k34wmqjsrcr2fdh0000gp/T/am_test077575705/config.yml level=debug ts=2021-01-21T09:39:04.752Z caller=main.go:469 routePrefix=/ level=info ts=2021-01-21T09:39:04.753Z caller=main.go:489 msg=Listening address=127.0.0.1:50004 level=debug ts=2021-01-21T09:39:05.521Z caller=dispatch.go:138 component=dispatcher msg="Received alert" alert=test1[5ae90ff][active] level=debug ts=2021-01-21T09:39:06.523Z caller=dispatch.go:473 component=dispatcher aggrGroup="{}:{alertname="test1"}" msg=flushing alerts=[test1[5ae90ff][active]] level=debug ts=2021-01-21T09:39:06.526Z caller=notify.go:685 component=dispatcher receiver=default integration=webhook[0] msg="Notify success" attempts=1 level=debug ts=2021-01-21T09:39:06.526Z caller=notify.go:685 component=dispatcher receiver=default integration=webhook[1] msg="Notify success" attempts=1 level=debug ts=2021-01-21T09:39:07.526Z caller=dispatch.go:473 component=dispatcher aggrGroup="{}:{alertname="test1"}" msg=flushing alerts=[test1[5ae90ff][active]] level=debug ts=2021-01-21T09:39:08.530Z caller=dispatch.go:473 component=dispatcher aggrGroup="{}:{alertname="test1"}" msg=flushing alerts=[test1[5ae90ff][active]] level=debug ts=2021-01-21T09:39:09.531Z caller=dispatch.go:473 component=dispatcher aggrGroup="{}:{alertname="test1"}" msg=flushing alerts=[test1[5ae90ff][active]] level=debug ts=2021-01-21T09:39:09.532Z caller=notify.go:685 component=dispatcher receiver=default integration=webhook[1] msg="Notify success" attempts=1 level=debug ts=2021-01-21T09:39:09.532Z caller=notify.go:685 component=dispatcher receiver=default integration=webhook[0] msg="Notify success" attempts=1 level=debug ts=2021-01-21T09:39:10.533Z caller=dispatch.go:473 component=dispatcher aggrGroup="{}:{alertname="test1"}" msg=flushing alerts=[test1[5ae90ff][active]] level=info ts=2021-01-21T09:39:11.019Z caller=main.go:528 msg="Received SIGTERM, exiting gracefully..." level=debug ts=2021-01-21T09:39:11.019Z caller=cluster.go:539 component=cluster msg="leaving cluster" level=debug ts=2021-01-21T09:39:11.019Z caller=delegate.go:236 component=cluster received=NotifyLeave node=01EWJ4ZJCSG0953MEZDW06QXPM addr=127.0.0.1:50007 level=debug ts=2021-01-21T09:39:11.019Z caller=cluster.go:492 component=cluster msg="peer left" peer=01EWJ4ZJCSG0953MEZDW06QXPM send_test.go:217: collector "webhook":

    interval [6,6.5]
    ---
    - &{map[] 0001-01-01T00:00:00.000Z <nil> [] 2021-01-21T17:39:05.515+08:00 <nil> <nil> { map[alertname:test1]}}[1:]
      [ ✗ ]
    interval [2,2.5]
    ---
    - &{map[] 0001-01-01T00:00:00.000Z <nil> [] 2021-01-21T17:39:05.515+08:00 <nil> <nil> { map[alertname:test1]}}[1:]
      [ ✓ ]
    
    received:
    @ 2.010035187
    - &{map[] 0001-01-01T00:00:00.000Z <nil> [] 2021-01-21T17:39:05.521+08:00 <nil> <nil> { map[alertname:test1]}}[1.005199:]
    @ 5.016554237
    - &{map[] 0001-01-01T00:00:00.000Z <nil> [] 2021-01-21T17:39:05.521+08:00 <nil> <nil> { map[alertname:test1]}}[1.005199:]
    
send_test.go:217: 
    collector "webhook_failing":
    
    interval [6,6.5]
    ---
    - &{map[] 0001-01-01T00:00:00.000Z <nil> [] 2021-01-21T17:39:05.515+08:00 <nil> <nil> { map[alertname:test1]}}[1:]
      [ ✗ ]
    
    received:
    @ 5.016386806
    - &{map[] 0001-01-01T00:00:00.000Z <nil> [] 2021-01-21T17:39:05.521+08:00 <nil> <nil> { map[alertname:test1]}}[1.005199:]

FAIL FAIL github.com/prometheus/alertmanager/test/with_api_v2/acceptance 26.649s ok github.com/prometheus/alertmanager/types (cached) ? github.com/prometheus/alertmanager/ui [no test files] FAIL make: *** [common-test] Error 1

  • System information: Darwin 19.6.0 x86_64

  • Alertmanager version: v0.21

ryanwuer avatar Jan 21 '21 09:01 ryanwuer

In my experience, those tests are a bit flaky at times. Do you get this error consistently if you retry running make test?

Help to make these tests less flaky is certainly welcome.

beorn7 avatar Jan 21 '21 12:01 beorn7

Yeah, i got this error consistently. Never succeed.

ryanwuer avatar Jan 22 '21 01:01 ryanwuer

Perhaps this test is reliably failing on Darwin only? Anyway, help appreciated…

beorn7 avatar Jan 22 '21 10:01 beorn7