PulsarIOTest.testReadFromSimpleTopic is very flaky

[damccorm]

PulsarIOTest uses shared client across all tests [1], which is a race conditions when multiple tests run in parallel.  This manifests as frequent test failures.

[1] https://github.com/apache/beam/blob/master/sdks/java/io/pulsar/src/test/java/org/apache/beam/sdk/io/pulsar/PulsarIOTest.java#L62

Imported from Jira BEAM-14269. Original Jira may contain additional context. Reported by: SteveNiemitz.

[damccorm]

Unable to assign user @MarcoRob. If able, self-assign, otherwise tag @damccorm so that he can assign you. Because of GitHub's spam prevention system, your activity is required to enable assignment in this repo.

[MarcoRob]

Hi @damccorm I am unable to self-assign, can you help me assign this issue to me? Thanks!

[damccorm]

Sure! As of https://github.com/apache/beam/pull/21719 you should be able to self assign with the command .take-issue though, so you should be able to do that going forward! Let me know if you run into any issues with that

[pabloem]

@MarcoRob are you working on this?

[Abacn]

@pabloem There was a draft PR #17473 seems not having updates for a while.

[pabloem]

hm from #17473 it seems more complex, huh? we'll see what I can do...

[Abacn]

hm from #17473 it seems more complex, huh? we'll see what I can do...

That seems right. It indicates issues for the IO connector itself.

[MarcoRob]

@MarcoRob are you working on this?

Hi @pabloem Yes I am working on it, just right now I am working on the migration of Github Actions (GA)

I consulted with Pulsar community and seems like the issue comes from the function that seeks(timestamp) the message from a timestamp it fails sometimes in retrieving the right message with the correct timestamp, so they advice me to use seek(MessageId) instead. So I am working on changing the function, but right now I paused this while GA migration is finished.

You can see I got some inputs on that issue in this PR-17473

[pabloem]

gotcha. thanks @MarcoRob !

[kennknowles]

Here another occurrence: https://ci-beam.apache.org/job/beam_PreCommit_Java_Commit/24746/

[apilloud]

Still happening: https://ci-beam.apache.org/job/beam_PreCommit_Java_Phrase/5782/consoleFull

14:56:31 > Task :sdks:java:io:pulsar:test
14:56:31 
14:56:31 org.apache.beam.sdk.io.pulsar.PulsarIOTest > testReadFromSimpleTopic FAILED
14:56:31     org.apache.beam.sdk.Pipeline$PipelineExecutionException at PulsarIOTest.java:178
14:56:31         Caused by: java.lang.IllegalArgumentException at Preconditions.java:440

[kennknowles]

@MarcoRob are you no longer working on this?

[kennknowles]

I'm going to unassign this for now, then. Feel free to comment or grab it again.