kafka-go icon indicating copy to clipboard operation
kafka-go copied to clipboard

Make `(msg *Message) totalSize()` public

Open taguhoiya opened this issue 1 year ago • 0 comments

Suggestion

  • I would like to make totalSize() public to get to know the exact size of kakfa.Message
  • now the method of knowing the message size is the package-private one and too complicated to implement on the client side.
  • (I actually used totalSize() in my repository and had to copy and paste many methods all the way)

can we make totalSize() public?

reference

here are all the methods that I personally implemented and I copied and pasted all methods literally from the https://github.com/segmentio/kafka-go/blob/main/message.go

source code
package kafkas

import (
	"github.com/segmentio/kafka-go"
)

// This file is all implemented by https://github.com/segmentio/kafka-go/blob/main/message.go
const timestampSize = 8

func varIntLen(i int64) int {
	u := uint64((i << 1) ^ (i >> 63)) // zig-zag encoding
	n := 0

	for u >= 0x80 {
		u >>= 7
		n++
	}

	return n + 1
}

func varBytesLen(b []byte) int {
	return varIntLen(int64(len(b))) + len(b)
}

func varStringLen(s string) int {
	return varIntLen(int64(len(s))) + len(s)
}

func varArrayLen(n int, f func(int) int) int {
	size := varIntLen(int64(n))
	for i := 0; i < n; i++ {
		size += f(i)
	}
	return size
}

func sizeofBytes(b []byte) int32 {
	return 4 + int32(len(b))
}

func size(msg *kafka.Message) int32 {
	return 4 + 1 + 1 + sizeofBytes(msg.Key) + sizeofBytes(msg.Value) + timestampSize
}

func headerSize(msg *kafka.Message) int {
	return varArrayLen(len(msg.Headers), func(i int) int {
		h := &msg.Headers[i]
		return varStringLen(h.Key) + varBytesLen(h.Value)
	})
}

// Make it public for external packages
func TotalSize(msg *kafka.Message) int32 {
	return int32(headerSize(msg)) + size(msg)
}

taguhoiya avatar Jan 09 '24 08:01 taguhoiya