Flaky tests list

[johanbrandhorst]

This issue tracks tests that are known to be flaky.

• TestServer_ReloadWorkerTags in internal/cmd/commands/server This will sometimes fail like this:

  === CONT  TestServer_ReloadWorkerTags
     worker_tags_reload_test.go:166: 
           Error Trace:	worker_tags_reload_test.go:146
                                   worker_tags_reload_test.go:166
           Error:      	Not equal: 
                          expected: "foo"
                          actual  : "type"
  
                          Diff:
                          --- Expected
                          +++ Actual
                          @@ -1 +1 @@
                          -foo
                          +type
           Test:       	TestServer_ReloadWorkerTags
  FAIL
  FAIL	github.com/hashicorp/boundary/internal/cmd/commands/server	84.648s
  

  Failing log. The exact cause is unknown at this time, but may be related to the amount of time spent sleeping in the test (sometimes it may not be long enough).
• Test_statusCodeInterceptor/nil-err in internal/servers/controller

  testing.go:802: 
           Error Trace:	testing.go:802
                                   testing.go:912
                                   testing.go:1049
                                   testing.go:1253
           Error:      	Received unexpected error:
                          closed
           Test:       	Test_statusCodeInterceptor/nil-err
  === RUN   Test_statusCodeInterceptor/hello
  --- FAIL: Test_statusCodeInterceptor (0.00s)
     --- PASS: Test_statusCodeInterceptor/nil-nil (0.00s)
     --- FAIL: Test_statusCodeInterceptor/nil-err (0.00s)
  

  Failing log.
• TestServer_ShutdownWorker in internal/cmd/commands/server

  worker_shutdown_reload_test.go:148: 
           Error Trace:	testing_helper.go:138
                                   testing_helper.go:312
                                   worker_shutdown_reload_test.go:148
           Error:      	Received unexpected error:
                          failed to read protobuf message: failed to get reader: received close frame: status = StatusProtocolError and reason = "received header with unexpected rsv bits set: false:false:true"
           Test:       	TestServer_ShutdownWorker
  --- FAIL: TestServer_ShutdownWorker (1.78s)
  

  Failing log.
• TestServer_ReloadWorkerTags in internal/cmd/commands/server

  === CONT  TestServer_ReloadWorkerTags
     worker_tags_reload_test.go:136: timeout
  --- FAIL: TestServer_ReloadWorkerTags (15.94s)
  

  Failing log.
• ~TestList in internal/tests/api/targets~

  - -- FAIL: TestList (1.46s)
  - panic: interface conversion: interface {} is nil, not float64 [recovered]
  -    panic: interface conversion: interface {} is nil, not float64
  - 
  - goroutine 4365 [running]:
  - testing.tRunner.func1.2({0x147b320, 0xc002ff69c0})
  -    /home/circleci/.go/src/testing/testing.go:1389 +0x24e
  - testing.tRunner.func1()
  -    /home/circleci/.go/src/testing/testing.go:1392 +0x39f
  - panic({0x147b320, 0xc002ff69c0})
  -    /home/circleci/.go/src/runtime/panic.go:838 +0x207
  - github.com/hashicorp/boundary/internal/tests/api/targets_test.TestList(0xc002f20820)
  -    /home/circleci/boundary/internal/tests/api/targets/target_test.go:193 +0x174b
  - testing.tRunner(0xc002f20820, 0x6043910)
  -    /home/circleci/.go/src/testing/testing.go:1439 +0x102
  - created by testing.(*T).Run
  -    /home/circleci/.go/src/testing/testing.go:1486 +0x35f
  - FAIL	github.com/hashicorp/boundary/internal/tests/api/targets	6.807s
  

  ~Failing log~ Resolved in #2086

[irenarindos]

Adding TestServer_ReloadListener in internal/cmd/commands/server to the list

listener_reload_test.go:151: timeout
--- FAIL: TestServer_ReloadListener (16.47s)

Failing log.

[kheina]

adding TestSessionCleanup/default/multi_controller in boundary/internal/tests/cluster

test output

--- FAIL: TestSessionCleanup (0.00s)
                                    
    --- FAIL: TestSessionCleanup/default (0.00s)
                                                
        --- FAIL: TestSessionCleanup/default/multi_controller (8.76s)
                                                                     
    session_cleanup_test.go:310: 
        	Error Trace:	testing_helper.go:138
        	            				testing_helper.go:312
        	            				session_cleanup_test.go:310
        	Error:      	Received unexpected error:
        	            	failed to read protobuf message: failed to get reader: received close frame: status = StatusProtocolError and reason = "received header with unexpected rsv bits set: false:false:true"
        	Test:       	TestSessionCleanup/default/multi_controller
{"id":"7C60PW6OLQ","source":"https://hashicorp.com/boundary/ip-172-28-5-152/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"worker.(Worker).handleProxy","data":{"connection_id":"sc_qqxJ1RVFuv","msg":"connection successfully authorized","session_id":"s_fhalODu8iv"}},"datacontentype":"application/cloudevents","time":"2022-08-12T14:52:56.350565028Z"}
{"id":"l6ZVZ7SQb9","source":"https://hashicorp.com/boundary/ip-172-28-5-152/worker","specversion":"1.0","type":"system","data":{"version":"v0.1","op":"cluster.testWorkerSessionCleanupMulti","data":{"msg":"running initial send/recv test"}},"datacontentype":"application/cloudevents","time":"2022-08-12T14:52:56.350708721Z"}

complete log output

[hugoghx]

Adding TestServer_ReloadWorkerTags in internal/cmd/commands/server/worker_tags_reload_test.go

Output

--- FAIL: TestServer_ReloadWorkerTags (31.65s)
                                              
    worker_tags_reload_test.go:131: 
        	Error Trace:	/home/circleci/boundary/internal/cmd/commands/server/worker_tags_reload_test.go:113
        	            				/home/circleci/boundary/internal/cmd/commands/server/worker_tags_reload_test.go:131
        	Error:      	Should be true
        	Test:       	TestServer_ReloadWorkerTags

[moduli]

Tests that failed due to failed to read protobuf message: failed to get reader: received close frame: status = StatusProtocolError and reason = "received header with unexpected rsv bits set: false:false:true" are hopefully resolved by this PR: https://github.com/hashicorp/boundary/pull/2584

[moduli]

@johanbrandhorst I haven't seen flaky unit tests in some time. I propose to close this issue and if we see failures in the future, I will file JIRA tickets for them and triage them there. Thoughts?

[johanbrandhorst]

Thanks for your work on fixing the flaky tests Mike, I agree we can close this umbrella issue :).