zio-keeper icon indicating copy to clipboard operation
zio-keeper copied to clipboard

Failing tests

Open klangner opened this issue 5 years ago • 5 comments

Hi, I just cloned this project and was trying to execute tests by running:

sbt
test

But it looks some of them fail on my computer:

Failed tests:
[error] 	zio.keeper.membership.SwimSpec
[error] 	zio.keeper.protocol.InternalProtocolSpec
[error] 	zio.keeper.transport.TransportSpec
[error] 	zio.keeper.membership.GossipStateSpec
[error] (keeper / Test / test) sbt.TestsFailedException: Tests unsuccessful
[error] (membership / Test / test) sbt.TestsFailedException: Tests unsuccessful

Should I run them in different way?

klangner avatar Jan 23 '20 09:01 klangner

No, you ran them correctly, but some of these are flaky as far as I know (@pshemass @mschuwalow). If you are interested, maybe you can dig into the root cause and investigate and/or fix some of the problems?

mijicd avatar Jan 23 '20 12:01 mijicd

I'm not super familiar with the current state of the specs in the main project. The membership ones are nonflaky, but I didn't have time to merge that part back in yet.

We definitely should spend some time to clean up and converge everything I'll try to get some progress there this weekend

mschuwalow avatar Jan 23 '20 15:01 mschuwalow

this looks very suspicious. zio.keeper.membership.GossipStateSpec is not flaky for sure

pshemass avatar Jan 23 '20 16:01 pshemass

Yes, flaky tests. I can't even replicate them now when I want to :-) I was only able to get them fail once, and only 1 failed:

[info] - cluster
[info]   - all nodes should have references to each other
[info]     List(NodeId(ff88effc-7a43-460a-b1da-4414371f7e37)) did not satisfy equalTo(List(NodeId(ff88effc-7a43-460a-b1da-4414371f7e37), NodeId(2dd1dacd-d3af-4c79-89e8-13dae0510394)))
[info]     List(NodeId(ff88effc-7a43-460a-b1da-4414371f7e37)) did not satisfy equalTo(List(NodeId(ff88effc-7a43-460a-b1da-4414371f7e37), NodeId(4b831725-acf7-4879-8f70-11aee842b6c6)))
[info] Done
[error] Failed tests:
[error] 	zio.keeper.membership.SwimSpec

It was after

sbt clean
sbt test

but only once, now it runs ok.

klangner avatar Jan 23 '20 18:01 klangner

I would add FailureDetectionSpec. See this and this.

mijicd avatar Jul 19 '20 20:07 mijicd