[Bug]: Portable Spark 3 Streaming runner appears to have GBK bugs, cannot execute Redistribute composite correctly

[kennknowles]

What happened?

See e.g. https://ge.apache.org/s/fqs6k37b63yko/tests/task/:runners:spark:3:job-server:validatesPortableRunnerStreaming/details/org.apache.beam.sdk.transforms.RedistributeTest?top-execution=1

There is no output from the test.

Issue Priority

Priority: 2 (default)

Issue Components

• [ ] Component: Python SDK
• [ ] Component: Java SDK
• [ ] Component: Go SDK
• [ ] Component: Typescript SDK
• [ ] Component: IO connector
• [ ] Component: Beam YAML
• [ ] Component: Beam examples
• [ ] Component: Beam playground
• [ ] Component: Beam katas
• [ ] Component: Website
• [X] Component: Spark Runner
• [ ] Component: Flink Runner
• [ ] Component: Samza Runner
• [ ] Component: Twister2 Runner
• [ ] Component: Hazelcast Jet Runner
• [ ] Component: Google Cloud Dataflow Runner

[kennknowles]

I notice that the portable Spark runner sickbays tests that would probably catch this too:

excludeTestsMatching 'org.apache.beam.sdk.transforms.GroupByKeyTest$WindowTests'

[kennknowles]

Ah and the equivalent ReshuffleTest

[Zawa-ll]

.take-issue

[github-actions[bot]]

This issue has been marked as stale due to 150 days of inactivity. It will be closed in 30 days if no further activity occurs. If you think that’s incorrect or this issue still needs to be addressed, please simply write any comment. If closed, you can reopen the issue at any time. Thank you for your contributions.