Error decoding packet with MockBroker

[Mwea]

Versions

Please specify real version numbers or git SHAs, not just "Latest" since that changes fairly regularly.

Sarama	Kafka	Go
v1.27.1	mockedBroker	1.15

Configuration

What configuration values are you using for Sarama and Kafka?

		"api.version.request": false,
		"bootstrap.servers": broker0.Addr(),
		"group.id": "toto",

Logs

When filing an issue please provide logs from Sarama and Kafka if at all possible. You can set sarama.Logger to a log.Logger to capture Sarama debug output.

logs: CLICK ME

=== RUN   TestConsumerCKG
    mockbroker.go:363: kafka: error decoding packet: invalid length

Problem Description

Hey 👋 I wanted to try testing confluent-kafka-go library with your MockBroker, however when running following test:

package main

import (
	"fmt"	
	"testing"
	
	"github.com/Shopify/sarama"
	"github.com/confluentinc/confluent-kafka-go/kafka"
)

// If a particular offset is provided then messages are consumed starting from
// that offset.
func TestConsumerCKG(t *testing.T) {
	// Given
	broker0 := sarama.NewMockBroker(t, 0)

	configMap := &kafka.ConfigMap{
		"api.version.request": false,
		"bootstrap.servers": broker0.Addr(),
		"group.id": "toto",
	}

	consumer, err := kafka.NewConsumer(configMap)
	if err != nil {
		return
	}
	var topic = "my_topic"
	err = consumer.Subscribe(topic, nil)
	if err != nil {
		return
	}

	err = consumer.Assign([]kafka.TopicPartition{kafka.TopicPartition{
		Topic:     &topic,
		Partition: 1,
		Offset:    kafka.Offset(-1),
		Metadata:  nil,
		Error:     nil,
	}})
	if err != nil {
		return
	}
	e := consumer.Poll(1000)
	if e != nil {
		fmt.Println("Hola")
	}
}

I encountered this issue:

    mockbroker.go:363: kafka: error decoding packet: invalid length

From my early investigation, it seems that it comes from this code:

	helper := realDecoder{raw: buf}
	err := in.decode(&helper)
	if err != nil {
		return err
	}

	if helper.off != len(buf) {
		return PacketDecodingError{"invalid length"}
	}

Where helper.off ends up being shorter than buf, especially, the []byte received is:

%08b [0 18 0 3 0 0 0 1 0 7 114 100 107 97 102 107 97 0 19 99 111 110 102 108 117 101 110 116 45 107 97 102 107 97 45 103 111 6 49 46 52 46 50 0]

I believe this is related to the implementation of in.decode(&helper) but I have to admit I'd be happy to get a hand on this as I didnt manage to find what could be wrong about the implementation.

Does it ring any bell to you ? please let me know if you need additional information 🙇

[Mwea]

Found out it comes from not implementing correctly https://cwiki.apache.org/confluence/display/KAFKA/KIP-511%3A+Collect+and+Expose+Client%27s+Name+and+Version+in+the+Brokers in mockedBroker , will give it a look and try to implement it

[dearchap]

@Mwea I'm getting the exact same error. Any ideas whats causing this ?

[dragonfriend0013]

@Mwea I am getting this same error, using ssarama throughout. github.com/Shopify/sarama v1.37.2

[github-actions[bot]]

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur. Please check if the main branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.

[github-actions[bot]]

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur. Please check if the main branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.