etcd icon indicating copy to clipboard operation
etcd copied to clipboard

Published 20 hours ago • verified publisher icon etcd-io

e2e: improve Watch coverage

Open hexfusion opened this issue 7 years ago • 10 comments

Incremental test coverage improvement for Watch.

Initial testing seems to fail during TestCtlV3AuthAndWatchInteractive.

ref: https://github.com/coreos/etcd/issues/9149

hexfusion avatar Mar 10 '18 20:03 hexfusion 

--- FAIL: TestCtlV3AuthAndWatchInteractive (15.70s)
	testutil.go:55: goroutine 719 [running]:
		github.com/coreos/etcd/pkg/testutil.FatalStack(0xc4202a01e0, 0xc4201693a0, 0x18)
			/home/samb/go/src/github.com/coreos/etcd/pkg/testutil/testutil.go:54 +0x74
		_/home/samb/d/go/etcd-test-flag/e2e.testCtl(0xc4202a01e0, 0xd30690, 0xc42007af90, 0x1, 0x1)
			/home/samb/d/go/etcd-test-flag/e2e/ctl_v3_test.go:184 +0x47e
		_/home/samb/d/go/etcd-test-flag/e2e.TestCtlV3AuthAndWatchInteractive(0xc4202a01e0)
			/home/samb/d/go/etcd-test-flag/e2e/ctl_v3_auth_test.go:49 +0x6a
		testing.tRunner(0xc4202a01e0, 0xd2ff48)
			/usr/lib/go/src/testing/testing.go:777 +0xd0
		created by testing.(*T).Run
			/usr/lib/go/src/testing/testing.go:824 +0x2e0
		
		goroutine 1 [chan receive]:
		testing.(*T).Run(0xc4202a01e0, 0xd10f21, 0x20, 0xd2ff48, 0x48bc01)
			/usr/lib/go/src/testing/testing.go:825 +0x301
		testing.runTests.func1(0xc4202a00f0)
			/usr/lib/go/src/testing/testing.go:1063 +0x64
		testing.tRunner(0xc4202a00f0, 0xc420299da8)
			/usr/lib/go/src/testing/testing.go:777 +0xd0
		testing.runTests(0xc42017cb40, 0x11a47c0, 0xe0, 0xe0, 0xc420186317)
			/usr/lib/go/src/testing/testing.go:1061 +0x2c4
		testing.(*M).Run(0xc4202aa000, 0x0)
			/usr/lib/go/src/testing/testing.go:978 +0x171
		_/home/samb/d/go/etcd-test-flag/e2e.TestMain(0xc4202aa000)
			/home/samb/d/go/etcd-test-flag/e2e/main_test.go:52 +0x4d5
		main.main()
			_testmain.go:486 +0x151
		
		goroutine 11 [chan receive]:
		github.com/coreos/etcd/pkg/logutil.(*MergeLogger).outputLoop(0xc420272200)
			/home/samb/go/src/github.com/coreos/etcd/pkg/logutil/merge_logger.go:173 +0x40d
		created by github.com/coreos/etcd/pkg/logutil.NewMergeLogger
			/home/samb/go/src/github.com/coreos/etcd/pkg/logutil/merge_logger.go:91 +0x85
		
		goroutine 722 [semacquire]:
		sync.runtime_notifyListWait(0xc420229510, 0x1)
			/usr/lib/go/src/runtime/sema.go:510 +0x10b
		sync.(*Cond).Wait(0xc420229500)
			/usr/lib/go/src/sync/cond.go:56 +0x80
		github.com/coreos/etcd/pkg/expect.(*ExpectProcess).ExpectFunc(0xc4202559d0, 0xc4201718c0, 0xd9ff60, 0xc42000c1b8, 0xc420169340, 0x13)
			/home/samb/go/src/github.com/coreos/etcd/pkg/expect/expect.go:101 +0x49
		github.com/coreos/etcd/pkg/expect.(*ExpectProcess).Expect(0xc4202559d0, 0xd10cc4, 0x1f, 0x0, 0x0, 0xc420169340, 0x13)
			/home/samb/go/src/github.com/coreos/etcd/pkg/expect/expect.go:119 +0x62
		_/home/samb/d/go/etcd-test-flag/e2e.ctlV3WatchFailPerm(0xc4202a01e0, 0x0, 0x0, 0xc42025e080, 0xb, 0x0, 0x0, 0x0, 0x1, 0x0, ...)
			/home/samb/d/go/etcd-test-flag/e2e/ctl_v3_watch_test.go:245 +0x28c
		_/home/samb/d/go/etcd-test-flag/e2e.authTestWatch(0xc4202a01e0, 0x0, 0x0, 0xc42025e080, 0xb, 0x0, 0x0, 0x0, 0x1, 0x0, ...)
			/home/samb/d/go/etcd-test-flag/e2e/ctl_v3_auth_test.go:857 +0xaaa
		_/home/samb/d/go/etcd-test-flag/e2e.testCtl.func2(0xc420256d80, 0xd30690, 0xc4200ca240)
			/home/samb/d/go/etcd-test-flag/e2e/ctl_v3_test.go:175 +0x93
		created by _/home/samb/d/go/etcd-test-flag/e2e.testCtl
			/home/samb/d/go/etcd-test-flag/e2e/ctl_v3_test.go:173 +0x2e1
		
		goroutine 740 [syscall]:
		syscall.Syscall(0x0, 0xc, 0xc4200ce000, 0x1000, 0xc420275200, 0xc4202286c0, 0xc42007cb48)
			/usr/lib/go/src/syscall/asm_linux_amd64.s:18 +0x5
		syscall.read(0xc, 0xc4200ce000, 0x1000, 0x1000, 0xc420228601, 0x0, 0x0)
			/usr/lib/go/src/syscall/zsyscall_linux_amd64.go:749 +0x5f
		syscall.Read(0xc, 0xc4200ce000, 0x1000, 0x1000, 0x14, 0x0, 0x0)
			/usr/lib/go/src/syscall/syscall_unix.go:162 +0x49
		internal/poll.(*FD).Read(0xc42002f5e0, 0xc4200ce000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
			/usr/lib/go/src/internal/poll/fd_unix.go:153 +0x118
		os.(*File).read(0xc42000c1b8, 0xc4200ce000, 0x1000, 0x1000, 0x410749, 0xc420169360, 0x20)
			/usr/lib/go/src/os/file_unix.go:226 +0x4e
		os.(*File).Read(0xc42000c1b8, 0xc4200ce000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
			/usr/lib/go/src/os/file.go:107 +0x6a
		bufio.(*Reader).fill(0xc42007cf70)
			/usr/lib/go/src/bufio/bufio.go:100 +0x11e
		bufio.(*Reader).ReadSlice(0xc42007cf70, 0x41070a, 0x14, 0xc420169360, 0xc42007cde0, 0x410749, 0xc4202835d0)
			/usr/lib/go/src/bufio/bufio.go:341 +0x2c
		bufio.(*Reader).ReadBytes(0xc42007cf70, 0xc42008af0a, 0x4, 0xc42008af00, 0xc42008af10, 0xc420229518, 0xc42007ce48)
			/usr/lib/go/src/bufio/bufio.go:419 +0x6b
		bufio.(*Reader).ReadString(0xc42007cf70, 0xa, 0x0, 0x0, 0x1, 0xc4202835d0)
			/usr/lib/go/src/bufio/bufio.go:459 +0x38
		github.com/coreos/etcd/pkg/expect.(*ExpectProcess).read(0xc4202559d0)
			/home/samb/go/src/github.com/coreos/etcd/pkg/expect/expect.go:78 +0x1ed
		created by github.com/coreos/etcd/pkg/expect.NewExpectWithEnv
			/home/samb/go/src/github.com/coreos/etcd/pkg/expect/expect.go:69 +0x215
		
		goroutine 721 [syscall]:
		syscall.Syscall(0x0, 0x6, 0xc4202af000, 0x1000, 0xc420275980, 0xc420228a40, 0xc42007bb48)
			/usr/lib/go/src/syscall/asm_linux_amd64.s:18 +0x5
		syscall.read(0x6, 0xc4202af000, 0x1000, 0x1000, 0xc420228a01, 0x0, 0x0)
			/usr/lib/go/src/syscall/zsyscall_linux_amd64.go:749 +0x5f
		syscall.Read(0x6, 0xc4202af000, 0x1000, 0x1000, 0x59, 0x0, 0x0)
			/usr/lib/go/src/syscall/syscall_unix.go:162 +0x49
		internal/poll.(*FD).Read(0xc42002e1e0, 0xc4202af000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
			/usr/lib/go/src/internal/poll/fd_unix.go:153 +0x118
		os.(*File).read(0xc42000c038, 0xc4202af000, 0x1000, 0x1000, 0x410749, 0xc420257ce0, 0x60)
			/usr/lib/go/src/os/file_unix.go:226 +0x4e
		os.(*File).Read(0xc42000c038, 0xc4202af000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
			/usr/lib/go/src/os/file.go:107 +0x6a
		bufio.(*Reader).fill(0xc42007bf70)
			/usr/lib/go/src/bufio/bufio.go:100 +0x11e
		bufio.(*Reader).ReadSlice(0xc42007bf70, 0x41070a, 0x59, 0xc420257ce0, 0x59, 0x59, 0x0)
			/usr/lib/go/src/bufio/bufio.go:341 +0x2c
		bufio.(*Reader).ReadBytes(0xc42007bf70, 0xc420257c0a, 0x59, 0x59, 0xc420257d40, 0x59, 0x0)
			/usr/lib/go/src/bufio/bufio.go:419 +0x6b
		bufio.(*Reader).ReadString(0xc42007bf70, 0xc42028840a, 0xc420257d40, 0x59, 0x0, 0x0)
			/usr/lib/go/src/bufio/bufio.go:459 +0x38
		github.com/coreos/etcd/pkg/expect.(*ExpectProcess).read(0xc420254cb0)
			/home/samb/go/src/github.com/coreos/etcd/pkg/expect/expect.go:78 +0x1ed
		created by github.com/coreos/etcd/pkg/expect.NewExpectWithEnv
			/home/samb/go/src/github.com/coreos/etcd/pkg/expect/expect.go:69 +0x215
		
	testutil.go:56: test timed out after 15s

hexfusion avatar Mar 10 '18 20:03 hexfusion

This seems to be an issue with the client not returning a "watch closed by server" message or a response at all, thus a timeout. Hope to review more tonight.

hexfusion avatar Mar 12 '18 12:03 hexfusion

/cc @jpbetz

wenjiaswe avatar Oct 15 '18 18:10 wenjiaswe

e2e test path has changed I will update this tonight and get it merged.

On Oct 16, 2018, at 12:58 PM, Joe Betz [email protected] wrote:

@jpbetz approved this pull request.

lgtm

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

hexfusion avatar Oct 16 '18 17:10 hexfusion

Codecov Report

Merging #9419 into master will increase coverage by 0.05%. The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #9419      +/-   ##
==========================================
+ Coverage   71.72%   71.77%   +0.05%     
==========================================
  Files         390      390              
  Lines       36346    36346              
==========================================
+ Hits        26068    26089      +21     
+ Misses       8470     8451      -19     
+ Partials     1808     1806       -2
Impacted Files Coverage Δ
etcdserver/api/v3rpc/watch.go 82.35% <0%> (-2.29%) :arrow_down:
clientv3/leasing/cache.go 87.77% <0%> (-1.12%) :arrow_down:
etcdserver/server.go 73.65% <0%> (-0.94%) :arrow_down:
clientv3/leasing/kv.go 89.03% <0%> (-0.67%) :arrow_down:
clientv3/balancer/grpc1.7-health.go 59.01% <0%> (-0.3%) :arrow_down:
mvcc/watchable_store.go 84.56% <0%> (+0.35%) :arrow_up:
etcdserver/v3_server.go 79.63% <0%> (+0.45%) :arrow_up:
etcdserver/raft.go 80.8% <0%> (+0.71%) :arrow_up:
pkg/netutil/netutil.go 70.49% <0%> (+0.81%) :arrow_up:
etcdctl/ctlv3/command/printer_simple.go 73.82% <0%> (+1.34%) :arrow_up:
... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 7a759c1...35c74e5. Read the comment docs.

codecov-io avatar Oct 17 '18 02:10 codecov-io

@hexfusion: Could you, please, rebase or close this PR ? Trying to cleanup old PRs and PR backlog.

ptabor avatar Jan 31 '21 22:01 ptabor

reminder....

/assign

hexfusion avatar Mar 17 '21 19:03 hexfusion

@hexfusion kindly ping. Let us try to get this merged :P

xiang90 avatar May 04 '21 20:05 xiang90

:) sorry about this. working on a few other things will clean this up.

hexfusion avatar May 05 '21 02:05 hexfusion

@hexfusion are you still working on this PR?

ahrtr avatar Aug 08 '22 22:08 ahrtr