java-sdk icon indicating copy to clipboard operation
java-sdk copied to clipboard

Migrate PubSub removing flaky test

Open mcruzdev opened this issue 5 months ago • 6 comments

Description

Use testcontainers for PubSub tests.

The main goal is to remove the following flaky test https://github.com/dapr/java-sdk/actions/runs/13773009141/job/38515768267#step:19:13432.

Closes one of the flaky test links related on the https://github.com/dapr/java-sdk/issues/1258 issue.

Issue reference

We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.

Please reference the issue this PR will close: #[issue number]

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

  • [x] Code compiles correctly
  • [x] Created/updated tests
  • [ ] Extended the documentation

mcruzdev avatar Jun 07 '25 04:06 mcruzdev

@artur-ciocanu @cicoyle @salaboy

Trying to move the following test to use Testcontainers I got a problem.

https://github.com/dapr/java-sdk/blob/3a8fd611da051a5e8bbb1e01a6a49bf348cda6c6/sdk-tests/src/test/java/io/dapr/it/pubsub/http/PubSubIT.java#L571-L572

When running the test from PubSub the following assert works well:

https://github.com/dapr/java-sdk/blob/3a8fd611da051a5e8bbb1e01a6a49bf348cda6c6/sdk-tests/src/test/java/io/dapr/it/pubsub/http/PubSubIT.java#L612

I have 1 message with the first message containing 10 statuses.

But using in-memory (Testcontainers) I have 10 messages and each one with 1 statuses.

I would like to know if the in-memory PubSub supports bulk message? The PubSub from DaprRun uses Redis

mcruzdev avatar Jun 07 '25 14:06 mcruzdev

@mcruzdev I will try to review ASAP, but could you please check why the ITs are failing?

artur-ciocanu avatar Jun 09 '25 16:06 artur-ciocanu

@mcruzdev I will try to review ASAP, but could you please check why the ITs are failing?

It is another mapped flaky test

mcruzdev avatar Jun 10 '25 02:06 mcruzdev

Error, I need to change the assert...

expected: 
  [io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@5bbe4e8f,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@dc7818,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@b2fdfa8,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@4613e544,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@abac76b4,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@e8f49fdd,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@bf3b3eb2,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@9843cc0f,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@3346a35b]
 but was: 
  [io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@f7ce0e8f,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@5bbe4e8f,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@dc7818,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@b2fdfa8,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@4613e544,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@abac76b4,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@e8f49fdd,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@bf3b3eb2,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@9843cc0f,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@3346a35b]

mcruzdev avatar Jun 24 '25 20:06 mcruzdev

@mcruzdev could you please check what is wrong with the tests, I see this error:

Error:    DaprPubSubIT.testLongValues:578 » Runtime org.opentest4j.AssertionFailedError: 
expected: 
  [io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@5bbe4e8f,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@dc7818,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@b2fdfa8,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@4613e544,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@abac76b4,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@e8f49fdd,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@bf3b3eb2,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@9843cc0f,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@3346a35b]
 but was: 
  [io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@f7ce0e8f,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@5bbe4e8f,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@dc7818,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@b2fdfa8,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@4613e544,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@abac76b4,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@e8f49fdd,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@bf3b3eb2,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@9843cc0f,
      io.dapr.it.pubsub.http.PubSubIT$ConvertToLong@3346a35b]

artur-ciocanu avatar Jun 24 '25 21:06 artur-ciocanu

Hi @artur-ciocanu we really need a custom ObjectSerializer, using a CustomizableObjectSerializer that allow the ObjectMapper be customizable are broking some tests.

I just copied the old behavior to the new way to write tests (using testcontainers to instantiate dapr).

mcruzdev avatar Jul 14 '25 19:07 mcruzdev

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 78.34%. Comparing base (d759c53) to head (3806ae6). Report is 179 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #1407      +/-   ##
============================================
+ Coverage     76.91%   78.34%   +1.42%     
- Complexity     1592     1878     +286     
============================================
  Files           145      234      +89     
  Lines          4843     5832     +989     
  Branches        562      601      +39     
============================================
+ Hits           3725     4569     +844     
- Misses          821      943     +122     
- Partials        297      320      +23     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov[bot] avatar Jul 14 '25 22:07 codecov[bot]