go icon indicating copy to clipboard operation
go copied to clipboard

services/ticker: TestInsertOrUpdateAsset random failure

Open bartekn opened this issue 6 years ago • 5 comments

TestInsertOrUpdateAsset failed when testing a commit that was a markdown file edit: https://circleci.com/gh/stellar/go/12298

bartekn avatar Dec 16 '19 17:12 bartekn

This test used to flake more frequently and I thought I fixed it by replacing use of Truncate with Round. That was in this commit: 3e2307012937199681852493b1b679a93689dbf5. The fix wasn't perfect, Postgres and Go round time differently:

The fix in this change isn't perfect according to discussion at lib/pq#227 (comment), where a commenter states that Postgres' rounding function rounds to nearest and in the event of a tie rounds to even, which is different to Go's Round function which rounds to nearest and in the event of a tie rounds away from zero.

I'll see if I can find a better fix.

The Go lib/pq issue relating to this is lib/pq#227.

leighmcculloch avatar Dec 16 '19 18:12 leighmcculloch

I've opened issue #2064 to add test logs to give us 👀 into what time inputs can be used to reproduce the issue. I won't attempt to fix the issue right now as the time cost to fix it without knowing what inputs it is failing on will be high. Next time it fails with these test logs report here and I'll fix it.

Hindsight: I wish I had added these test logs when I added the last fix.

leighmcculloch avatar Dec 16 '19 18:12 leighmcculloch

To my knowledge these tests haven't flaked since this issue was opened, or since logs were added in #2064. The logs are there to help us fix the issue if or when it happens again. I'll close this issue now. It can be reopened or a new issue created if this test flakes again.

leighmcculloch avatar Nov 10 '20 16:11 leighmcculloch

Run into this again: https://app.circleci.com/pipelines/github/stellar/go/8281/workflows/262b90cf-5920-4a0d-9f1a-fbac765c442b/jobs/44938

bartekn avatar May 31 '21 16:05 bartekn

Another instance: https://app.circleci.com/pipelines/github/stellar/go/9166/workflows/f31d23c8-2110-4bbc-a73b-3c39b2966e38/jobs/52124

bartekn avatar Aug 30 '21 15:08 bartekn