redpanda icon indicating copy to clipboard operation
redpanda copied to clipboard

Unsupported kafka api v3 (leave group) in `PartitionMovementUpgradeTest`.`test_basic_upgrade`

Open dotnwat opened this issue 2 years ago • 9 comments

https://buildkite.com/redpanda/redpanda/builds/13660#01826c94-abff-4f4f-ba8e-e64331d9b328

test_id:    rptest.tests.partition_movement_upgrade_test.PartitionMovementUpgradeTest.test_basic_upgrade
--
  | status:     FAIL
  | run time:   2 minutes 8.856 seconds
  |  
  |  
  | <BadLogLines nodes=docker-rp-23(20) example="ERROR 2022-08-05 07:01:42,886 [shard 0] rpc - server.cc:116 - kafka rpc protocol - Error[applying protocol] remote address: 172.16.32.22:49704 - std::runtime_error (Unsupported version 3 for leave group API)">
  | Traceback (most recent call last):
  | File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 135, in run
  | data = self.run_test()
  | File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 227, in run_test
  | return self.test_context.function(self.test)
  | File "/root/tests/rptest/services/cluster.py", line 48, in wrapped
  | self.redpanda.raise_on_bad_logs(allow_list=log_allow_list)
  | File "/root/tests/rptest/services/redpanda.py", line 1121, in raise_on_bad_logs
  | raise BadLogLines(bad_lines)
  | rptest.services.utils.BadLogLines: <BadLogLines nodes=docker-rp-23(20) example="ERROR 2022-08-05 07:01:42,886 [shard 0] rpc - server.cc:116 - kafka rpc protocol - Error[applying protocol] remote address: 172.16.32.22:49704 - std::runtime_error (Unsupported version 3 for leave group API)">

dotnwat avatar Aug 05 '22 14:08 dotnwat

Similar failure on the same test, on a different group API

FAIL test: PartitionMovementUpgradeTest.test_basic_upgrade (7/37 runs) failure at 2022-08-05T07:30:11.541Z: <BadLogLines nodes=docker-rp-18(17) example="ERROR 2022-08-05 06:43:21,331 [shard 0] rpc - server.cc:116 - kafka rpc protocol - Error[applying protocol] remote address: 172.16.16.5:38260 - std::runtime_error (Unsupported version 5 for join group API)"> in job https://buildkite.com/redpanda/redpanda/builds/13658#01826c85-9f41-44eb-9564-83cc5724d06e

jcsp avatar Aug 05 '22 14:08 jcsp

Another instance https://ci-artifacts.dev.vectorized.cloud/redpanda/01826f8f-f896-46fd-9a53-0a31b2af399b/vbuild/ducktape/results/2022-08-05--001/report.html

rystsov avatar Aug 06 '22 04:08 rystsov

This seems similar: https://ci-artifacts.dev.vectorized.cloud/redpanda/018279f4-00b6-44e0-956b-20f174bc3a6e/vbuild/ducktape/results/2022-08-07--001/PartitionMovementUpgradeTest/test_basic_upgrade/138/

[INFO  - 2022-08-07 21:30:41,423 - runner_client - log - lineno:278]: RunnerClient: rptest.tests.partition_movement_upgrade_test.PartitionMovementUpgradeTest.test_basic_upgrade: Summary: <BadLogLines nodes=docker-rp-6(20) example="ERROR 2022-08-07 21:29:42,458 [shard 0] rpc - server.cc:116 - kafka rpc protocol - Error[applying protocol] remote address: 172.16.16.9:54822 - std::runtime_error (Unsupported version 5 for join group API)">
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 135, in run
    data = self.run_test()
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 227, in run_test
    return self.test_context.function(self.test)
  File "/root/tests/rptest/services/cluster.py", line 48, in wrapped
    self.redpanda.raise_on_bad_logs(allow_list=log_allow_list)
  File "/root/tests/rptest/services/redpanda.py", line 1121, in raise_on_bad_logs
    raise BadLogLines(bad_lines)
rptest.services.utils.BadLogLines: <BadLogLines nodes=docker-rp-6(20) example="ERROR 2022-08-07 21:29:42,458 [shard 0] rpc - server.cc:116 - kafka rpc protocol - Error[applying protocol] remote address: 172.16.16.9:54822 - std::runtime_error (Unsupported version 5 for join group API)">

BenPope avatar Aug 08 '22 08:08 BenPope

Seen again - seems to be the same as the one I reported yesterday: https://buildkite.com/redpanda/redpanda/builds/13844#018281c1-1808-413b-b7e8-9e72e8d0082e

[INFO  - 2022-08-09 09:41:23,203 - runner_client - log - lineno:278]: RunnerClient: rptest.tests.partition_movement_upgrade_test.PartitionMovementUpgradeTest.test_basic_upgrade: FAIL: <BadLogLines nodes=docker-rp-20(35) example="ERROR 2022-08-09 09:40:44,134 [shard 0] rpc - server.cc:116 - kafka rpc protocol - Error[applying protocol] remote address: 172.16.16.7:44242 - std::runtime_error (Unsupported version 5 for join group API)">
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 135, in run
    data = self.run_test()
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 227, in run_test
    return self.test_context.function(self.test)
  File "/root/tests/rptest/services/cluster.py", line 48, in wrapped
    self.redpanda.raise_on_bad_logs(allow_list=log_allow_list)
  File "/root/tests/rptest/services/redpanda.py", line 1125, in raise_on_bad_logs
    raise BadLogLines(bad_lines)
rptest.services.utils.BadLogLines: <BadLogLines nodes=docker-rp-20(35) example="ERROR 2022-08-09 09:40:44,134 [shard 0] rpc - server.cc:116 - kafka rpc protocol - Error[applying protocol] remote address: 172.16.16.7:44242 - std::runtime_error (Unsupported version 5 for join group API)">

BenPope avatar Aug 09 '22 14:08 BenPope

Seen again - seems to be the same as the one I reported yesterday: https://ci-artifacts.dev.vectorized.cloud/redpanda/018281c1-1808-413b-b7e8-9e72e8d0082e/vbuild/ducktape/results/2022-08-09--001/PartitionMovementUpgradeTest/test_basic_upgrade/137/

[INFO  - 2022-08-09 09:41:23,203 - runner_client - log - lineno:278]: RunnerClient: rptest.tests.partition_movement_upgrade_test.PartitionMovementUpgradeTest.test_basic_upgrade: FAIL: <BadLogLines nodes=docker-rp-20(35) example="ERROR 2022-08-09 09:40:44,134 [shard 0] rpc - server.cc:116 - kafka rpc protocol - Error[applying protocol] remote address: 172.16.16.7:44242 - std::runtime_error (Unsupported version 5 for join group API)">
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 135, in run
    data = self.run_test()
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 227, in run_test
    return self.test_context.function(self.test)
  File "/root/tests/rptest/services/cluster.py", line 48, in wrapped
    self.redpanda.raise_on_bad_logs(allow_list=log_allow_list)
  File "/root/tests/rptest/services/redpanda.py", line 1125, in raise_on_bad_logs
    raise BadLogLines(bad_lines)
rptest.services.utils.BadLogLines: <BadLogLines nodes=docker-rp-20(35) example="ERROR 2022-08-09 09:40:44,134 [shard 0] rpc - server.cc:116 - kafka rpc protocol - Error[applying protocol] remote address: 172.16.16.7:44242 - std::runtime_error (Unsupported version 5 for join group API)">

@BenPope can you link to the buildkite job (not the artifacts) or the PR?

dotnwat avatar Aug 09 '22 18:08 dotnwat

@BenPope was that from a 22.1.x run? Perhaps it's just that the fix needs to be backported?

dotnwat avatar Aug 09 '22 18:08 dotnwat

@mmaslankaprv ^

dotnwat avatar Aug 09 '22 18:08 dotnwat

@BenPope can you link to the buildkite job (not the artifacts) or the PR?

Comment updated.

@BenPope was that from a 22.1.x run? Perhaps it's just that the fix needs to be backported?

It was dev

BenPope avatar Aug 09 '22 18:08 BenPope

I guess we are going to keep seeing this unless we backport the change

mmaslankaprv avatar Aug 09 '22 18:08 mmaslankaprv