Extend tests utility belt

[eiri]

It's be nice to have some standard test wrapper with 'retry until' or 'pass if any of N' functionality to deal with things like:

Compact after lowering attachment size limit couchdb_attachments_tests:544: should_compact_after_lowering_attachment_size_limit...*failed*

in function couchdb_attachments_tests:wait_compaction/3 (test/couchdb_attachments_tests.erl, line 586)
in call from couchdb_attachments_tests:compact_db/1 (test/couchdb_attachments_tests.erl, line 573)
in call from couchdb_attachments_tests:'-should_compact_after_lowering_attachment_size_limit/1-fun-1-'/1 (test/couchdb_attachments_tests.erl, line 550)

**error:{assertion_failed,[{module,couchdb_attachments_tests}, {line,573},
{reason,"Timeout waiting for database database compaction"}]}
  output:<<"">>

[janl]

@eiri problem is eunit standard timeouts. So even if we do a spinloop of “do until success”, we’d run into the eunit timeout. We can raise that of course, but we’ll be doing that a lot.

Maybe there is a way to set a timeout globally and set it to really long for travis and jenkins runs, so we never run into them™

[SinghCod3r]

@janl , i want to be part of it. As my core skill is tester so i want to look into this project. So if you could assign me then i will be able to part of it.

[SinghCod3r]

Hi @janl and @eiri,

Thank you for the discussion on this. I would like to work on this issue.

I've looked into the problem, and the assertion_failed error is indeed due to a race condition where the test checks for the database size before the asynchronous compaction has completed.

My proposed solution follows eiri's suggestion to create a standard test wrapper with retry logic.

1. Create a retry_until_success/3 helper function in a test utility module. This function will repeatedly execute a given function, catching any assertion errors and retrying after a short delay.
2. Refactor the wait_compaction/3 function in couchdb_attachments_tests.erl to use this new helper. This will make the test resilient to normal variations in compaction time.

This approach directly addresses the flakiness. Regarding @janl's point about eunit timeouts, the retry mechanism will have its own internal timeout (e.g., 50 retries with a 100ms delay = 5 seconds max), which should be sufficient for the compaction to complete without hitting the global test suite timeout.

I will start working on a PR with this implementation. Please assign this issue to me.

Thanks!

[janl]

we do not assign issues generally, you can open a PR if you want to contribute and reference this issue there. Please be sure to read our CONTRIBUTING guidelines