Make test suite run reliably locally

[hboon]

It used to be that it runs (more?) reliably locally, but not on Travis, but not anymore. Let's get it to work properly, locally first.

Not product priority, but it puts a dent on development (we can't rely on it for merge queues)

• [ ] Remove PublisherToAsyncTests.testCancellation
• [ ] more

[eviltofu]

Normally, I would not put in tests which access servers in the test suite. I would mock all network traffic.

[hboon]

mock all network traffic.

I don't think that's possible for us. It could be a good practice to mock some and reduce dependency. But ultimately we need to ensure the web APIs we call work as we expect.

Are the failures related to web APIs that we call though?

[eviltofu]

This morning's test failures: First run

Failing tests:
	EnsResolverTests.testEnsIp10WildcardAndEip3668CcipRead()
	EnsResolverTests.testEnsIp10WildcardAndEip3668CcipRead()
	EnsResolverTests.testResolution()
	EnsResolverTests.testResolution()
	EnsResolverTests.testResolutionThatHasDifferentOwnerAndResolver()
	EnsResolverTests.testResolutionThatHasDifferentOwnerAndResolver()

Second run

Failing tests:
	PublisherToAsyncTests.testCancellation()

[eviltofu]

mock all network traffic.

I don't think that's possible for us. It could be a good practice to mock some and reduce dependency. But ultimately we need to ensure the web APIs we call work as we expect.

Are the failures related to web APIs that we call though?

We can mock all the tests for logic. Then add just one test per network endpoint to make sure we have the correct data format to minimise the network traffic.

[hboon]

PublisherToAsyncTests.testCancellation

Thing is, this doesn't involve the network. The ENS ones, I suggest you look into why it's failing. They seem to work more reliably when they run standalone. It could still be due to network connectivity, but that would be horrible because our app should hit the networks much more aggresively than the tests.

[eviltofu]

What are we testing in testCancellation? @oa-s

[eviltofu]

Today's failures:

Failing tests:
	RetryPublisherTests.testRetryWithRandomDelay()
	RetryPublisherTests.testRetryWithRandomDelay()

[hboon]

@oa-s in case you missed this:

What are we testing in testCancellation? @oa-s

[oa-s]

What are we testing in testCancellation? @oa-s

that looks unfinished, should be apropriate handling cancellation for migration from publisher to async await functions

[eviltofu]

What are we testing in testCancellation? @oa-s

that looks unfinished, should be apropriate handling cancellation for migration from publisher to async await functions

@hboon should I remove this test?

[hboon]

Sure. I think there's a few others that might have timing issues though.

[eviltofu]

Can this be closed? The current tests on have failures in EtherKeystoreTests. @hboon

[hboon]

Hmm.. Let's keep it open since we should fix it ultimately, but you can unassign yourself.