f1 icon indicating copy to clipboard operation
f1 copied to clipboard

Fix flaky tests

Open nvloff-f3 opened this issue 1 year ago • 0 comments

  • [x] run tests without cache to reveal instability
    • [x] #184

Tests:

  • [x] Fix randomly failing TestParameterised/finishes_at_ends_of_duration
    run_stage_test.go:382: 
        	Error Trace:	/home/runner/work/f1/f1/internal/run/run_stage_test.go:382
        	            				/home/runner/work/f1/f1/internal/run/run_cmd_test.go:395
        	Error:      	Not equal: 
        	            	expected: false
        	            	actual  : true
        	Test:       	TestParameterised/finishes_at_ends_of_duration
        	Messages:   	command failed
--- FAIL: TestParameterised (2.11s)
  • [x] Fix randomly failing "TestRandomDistribution"
    run_stage_test.go:376: 
        	Error Trace:	/home/runner/work/f1/f1/internal/run/run_stage_test.go:376
        	            				/home/runner/work/f1/f1/internal/run/run_cmd_test.go:456
        	Error:      	"1" is not greater than "1"
        	Test:       	TestRandomDistribution
--- FAIL: TestRandomDistribution (0.50s)

https://github.com/form3tech-oss/f1/actions/runs/9153279187/job/25161972126?pr=240:

=== NAME  TestParameterised/simple_config_file_test
    run_stage_test.go:244: 
        	Error Trace:	/home/runner/work/f1/f1/internal/run/run_stage_test.go:244
        	            				/home/runner/work/f1/f1/internal/run/run_cmd_test.go:400
        	Error:      	Not equal: 
        	            	expected: 0x69
        	            	actual  : 0x6a
        	Test:       	TestParameterised/simple_config_file_test
        	Messages:   	number of started iterations
    run_stage_test.go:331: 
        	Error Trace:	/home/runner/work/f1/f1/internal/run/run_stage_test.go:331
        	            				/home/runner/work/f1/f1/internal/run/run_cmd_test.go:403
        	Error:      	Not equal: 
        	            	expected: 105
        	            	actual  : 106
        	Test:       	TestParameterised/simple_config_file_test
        	Messages:   	iteration teardown was not called expected times

nvloff-f3 avatar Mar 18 '24 03:03 nvloff-f3