pulsar-client-go
pulsar-client-go copied to clipboard
Published
20 hours ago •
apache
apache
gogo/protobuf is deprecated
Is your feature request related to a problem? Please describe.
Nowadays, the famous https://github.com/gogo/protobuf is deprecated and it is not compatible with all brand new golang protobuf features.
For instance, when using protobuf as schema, it prompts some errors like such:
Test Panicked
protobuf tag not enough fields in Record.state:
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/gogo/[email protected]/proto/table_unmarshal.go:341
click to see the full logs
Schema
/Users/hys/dev/go-workspace/src/go.planetmeican.com/nerds/mq/pulsar/schema_test.go:72
ProduceAndConsume
/Users/hys/dev/go-workspace/src/go.planetmeican.com/nerds/mq/pulsar/schema_test.go:92
with avro and json and protobuf schema
/Users/hys/dev/go-workspace/src/go.planetmeican.com/nerds/mq/pulsar/schema_test.go:100
should be able to consume the produced message [It]
/Users/hys/dev/go-workspace/src/go.planetmeican.com/nerds/mq/pulsar/schema_test.go:101
Test Panicked
protobuf tag not enough fields in Record.state:
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/gogo/[email protected]/proto/table_unmarshal.go:341
Full Stack Trace
github.com/gogo/protobuf/proto.(*unmarshalInfo).computeUnmarshalInfo(0xc0000ac000)
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/gogo/[email protected]/proto/table_unmarshal.go:341 +0x138a
github.com/gogo/protobuf/proto.(*unmarshalInfo).unmarshal(0xc0000ac000, {0x489e980}, {0xc00034c0de, 0x29, 0x29})
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/gogo/[email protected]/proto/table_unmarshal.go:138 +0x67
github.com/gogo/protobuf/proto.(*InternalMessageInfo).Unmarshal(0xc000436680, {0x2d7d7a40, 0xc0002ed740}, {0xc00034c0de, 0x29, 0x29})
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/gogo/[email protected]/proto/table_unmarshal.go:63 +0xd0
github.com/gogo/protobuf/proto.(*Buffer).Unmarshal(0xc00081ec78, {0x2d7d7a40, 0xc0002ed740})
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/gogo/[email protected]/proto/decode.go:424 +0x153
github.com/gogo/protobuf/proto.Unmarshal({0xc00034c0de, 0x29, 0x29}, {0x2d7d7a40, 0xc0002ed740})
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/gogo/[email protected]/proto/decode.go:342 +0xef
github.com/apache/pulsar-client-go/pulsar.(*ProtoSchema).Decode(0x6900f18, {0xc00034c0de, 0x29, 0x29}, {0x48896a0, 0xc0002ed740})
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/apache/[email protected]/pulsar/schema.go:149 +0x58
github.com/apache/pulsar-client-go/pulsar.(*message).GetSchemaValue(0x58, {0x48896a0, 0xc0002ed740})
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/apache/[email protected]/pulsar/impl_message.go:291 +0x4d
go.planetmeican.com/nerds/mq/pulsar_test.glob..func8.3.1.1()
/Users/hys/dev/go-workspace/src/go.planetmeican.com/nerds/mq/pulsar/schema_test.go:200 +0x134c
github.com/onsi/ginkgo/internal/leafnodes.(*runner).runSync(0xc00081f440)
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:113 +0xba
github.com/onsi/ginkgo/internal/leafnodes.(*runner).run(0x0)
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:64 +0x125
github.com/onsi/ginkgo/internal/leafnodes.(*ItNode).Run(0xc00081f760)
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/it_node.go:26 +0x7b
github.com/onsi/ginkgo/internal/spec.(*Spec).runSample(0xc00027fb30, 0xc00081faa8, {0x49f03c0, 0xc000450180})
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/onsi/[email protected]/internal/spec/spec.go:215 +0x2a9
github.com/onsi/ginkgo/internal/spec.(*Spec).Run(0xc00027fb30, {0x49f03c0, 0xc000450180})
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/onsi/[email protected]/internal/spec/spec.go:138 +0xe7
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runSpec(0xc0001fe280, 0xc00027fb30)
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/onsi/[email protected]/internal/specrunner/spec_runner.go:200 +0xe5
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runSpecs(0xc0001fe280)
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/onsi/[email protected]/internal/specrunner/spec_runner.go:170 +0x1a5
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).Run(0xc0001fe280)
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/onsi/[email protected]/internal/specrunner/spec_runner.go:66 +0xc5
github.com/onsi/ginkgo/internal/suite.(*Suite).Run(0xc00043a2a0, {0x2d8ad520, 0xc000102b60}, {0x48e4ebe, 0x4d27b20}, {0xc000118030, 0x1, 0x1}, {0x4a06f58, 0xc000450180}, ...)
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/onsi/[email protected]/internal/suite/suite.go:79 +0x4d2
github.com/onsi/ginkgo.RunSpecsWithCustomReporters({0x49f1180, 0xc000102b60}, {0x48e4ebe, 0xc}, {0xc00013b718, 0x1, 0xc00013b728})
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/onsi/[email protected]/ginkgo_dsl.go:229 +0x185
github.com/onsi/ginkgo.RunSpecs({0x49f1180, 0xc000102b60}, {0x48e4ebe, 0xc})
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/onsi/[email protected]/ginkgo_dsl.go:210 +0x10b
go.planetmeican.com/nerds/mq/pulsar_test.TestPulsar(0x0)
/Users/hys/dev/go-workspace/src/go.planetmeican.com/nerds/mq/pulsar/pulsar_suite_test.go:23 +0x85
testing.tRunner(0xc000102b60, 0x4921b20)
/Users/hys/.gvm/gos/go1.17/src/testing/testing.go:1259 +0x102
created by testing.(*T).Run
/Users/hys/.gvm/gos/go1.17/src/testing/testing.go:1306 +0x35a
Furthermore, using the google.golang.org/protobuf/reflect/protoreflect and the google.golang.org/protobuf/types/dynamicpb
to create dynamic protobuf message also causes trouble.
For example, gogo/protobuf has the checking tags issue: https://github.com/gogo/protobuf/issues/678
protobuf tag not enough fields in Message.typ:
/Users/hys/.gvm/pkgsets/go1.17/global/pkg/mod/github.com/gogo/[email protected]/proto/table_unmarshal.go:341
Example code to create dynamic object
getProtoMsgDescriptor := func() protoreflect.MessageDescriptor {
fddef := &descriptorpb.FileDescriptorProto{
Name: proto.String("test.proto"),
Package: proto.String("test"),
MessageType: []*descriptorpb.DescriptorProto{
{
Name: proto.String("Record"),
Field: []*descriptorpb.FieldDescriptorProto{
{
Name: proto.String("snowflake_id"),
JsonName: proto.String("snowflake_id,omitempty"),
Number: proto.Int32(1),
Type: descriptorpb.FieldDescriptorProto_Type(protoreflect.StringKind).Enum(),
},
{
Name: proto.String("occur_timestamp"),
JsonName: proto.String("occur_timestamp,omitempty"),
Number: proto.Int32(2),
Type: descriptorpb.FieldDescriptorProto_Type(protoreflect.Int64Kind).Enum(),
},
{
// usually we use google.protobuf.StringValue to do nullable string,
// but use dynamic message it is too much painful to include it into the resolver scope.
// so we'll instead use plain string.
Name: proto.String("from_system"),
JsonName: proto.String("from_system,omitempty"),
Number: proto.Int32(3),
Type: descriptorpb.FieldDescriptorProto_Type(protoreflect.StringKind).Enum(),
},
{
Name: proto.String("action_type"),
JsonName: proto.String("action_type,omitempty"),
Number: proto.Int32(4),
Type: descriptorpb.FieldDescriptorProto_Type(protoreflect.StringKind).Enum(),
},
},
},
},
Syntax: proto.String("proto3"),
}
fd, err := protodesc.NewFile(fddef, nil)
Expect(err).NotTo(HaveOccurred(), "%+v", err)
md := fd.Messages().ByName("Record")
return md
}
Describe the solution you'd like
My suggestion would be to replace all gogo/protobuf usage by the official google.golang.org/protobuf.
Per my understanding, this may require a re-generation of all generated pb.go files and some imports.
Thanks for your attention.
