danielgtaylor
Struct cannot be serialized anymore
Summary
Struct().from_dict({'a':'a'}).SerializeToString() raises TypeError: string argument without an encoding
Reproduction Steps
from betterproto.lib.google.protobuf import Struct
Struct().from_dict({'a':'a'}).SerializeToString()
Expected Results
no exception raised and proper serialization bytes returned.
Actual Results
raises TypeError: string argument without an encoding with traceback:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Cell In[10], line 1
----> 1 Struct().from_dict({'a':'a'}).SerializeToString()
File [...]/.venv/lib/python3.12/site-packages/betterproto/__init__.py:1138, in Message.SerializeToString(self)
1125 def SerializeToString(self: T) -> bytes:
1126 """
1127 Get the binary encoded Protobuf representation of this message instance.
1128
(...)
1136 The binary encoded Protobuf representation of this message instance
1137 """
-> 1138 return bytes(self)
File [...]/.venv/lib/python3.12/site-packages/betterproto/__init__.py:1026, in Message.__bytes__(self)
1022 """
1023 Get the binary encoded Protobuf representation of this message instance.
1024 """
1025 with BytesIO() as stream:
-> 1026 self.dump(stream)
1027 return stream.getvalue()
File [...]/.venv/lib/python3.12/site-packages/betterproto/__init__.py:993, in Message.dump(self, stream, delimit)
991 assert meta.map_types
992 sk = _serialize_single(1, meta.map_types[0], k)
--> 993 sv = _serialize_single(2, meta.map_types[1], v)
994 stream.write(
995 _serialize_single(meta.number, meta.proto_type, sk + sv)
996 )
997 else:
998 # If we have an empty string and we're including the default value for
999 # a oneof, make sure we serialize it. This ensures that the byte string
1000 # output isn't simply an empty string. This also ensures that round trip
1001 # serialization will keep `which_one_of` calls consistent.
File [...]/.venv/lib/python3.12/site-packages/betterproto/__init__.py:464, in _serialize_single(field_number, proto_type, value, serialize_empty, wraps)
455 def _serialize_single(
456 field_number: int,
457 proto_type: str,
(...)
461 wraps: str = "",
462 ) -> bytes:
463 """Serializes a single field and value."""
--> 464 value = _preprocess_single(proto_type, wraps, value)
466 output = bytearray()
467 if proto_type in WIRE_VARINT_TYPES:
File [...]/.venv/lib/python3.12/site-packages/betterproto/__init__.py:415, in _preprocess_single(proto_type, wraps, value)
412 return b""
413 value = _get_wrapper(wraps)(value=value)
--> 415 return bytes(value)
417 return value
TypeError: string argument without an encoding
System Information
$ protoc --version; python --version; pip show betterproto
libprotoc 27.1
Python 3.12.4
Name: betterproto
Version: 2.0.0b7
Summary: A better Protobuf / gRPC generator & library
Home-page: https://github.com/danielgtaylor/python-betterproto
Author: Daniel G. Taylor
Author-email: [email protected]
License: MIT
Location: [...]/.venv/lib/python3.12/site-packages
Requires: grpclib, python-dateutil, typing-extensions
Required-by: [...]
Checklist
- [X] I have searched the issues for duplicates.
- [X] I have shown the entire traceback, if possible.
- [X] I have verified this issue occurs on the latest prelease of betterproto which can be installed using
pip install -U --pre betterproto, if possible.
Getting an issue with Struct Serialization as well, but from another place (seems like the Rust codec is the cause):
In [8]: s = Struct().from_dict({"a": 1})
In [9]: s.SerializeToString()
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
Cell In[9], line 1
----> 1 s.SerializeToString()
File ~/.local/lib/python3.10/site-packages/betterproto/__init__.py:1138, in Message.SerializeToString(self)
1125 def SerializeToString(self: T) -> bytes:
1126 """
1127 Get the binary encoded Protobuf representation of this message instance.
1128
(...)
1136 The binary encoded Protobuf representation of this message instance
1137 """
-> 1138 return bytes(self)
File ~/.local/lib/python3.10/site-packages/betterproto/__init__.py:1890, in __bytes_patch(self)
1889 def __bytes_patch(self) -> bytes:
-> 1890 return betterproto_rust_codec.serialize(self)
RuntimeError: Given object is not a valid betterproto message.
The issue @leonardgerardatomicmachinescom described is shown when Rust codec isn't installed.
@zacharya19 what does one need to install?
Assuming you got to the exception I got, you need to install betterproto >= 2.0.7b to solve it.
As I mentioned, this still doesn't work with the Rust codec and the current versions (betterproto-rust-codec 0.1.1, betterproto 2.0.7b), so make sure you don't have the rust codec installed.
I had to drop the struct usage due to bad performance.
I am getting the original error annd am on 2.0.7b I use betterproto[compiler] (cant generate code without it i think - but i dont think i have the rust-codec installed)
I am seeing some similar issue (server side when attempting to serialize a response message that contains a List[str].
I do not have the rust codec installed, installing it causes it to throw a different error:
"Given object is not a valid betterproto message."
It is not easy (or doesn't appear to be to me) to debug messages streamed as a AsyncIterator so I am not sure what fields in the response message are failing here and guessing from some print statements added to the betterproto/init.py and enabling the GRPCLib logs
[2025-07-11 15:17:53,523] [ERROR ] grpclib.server: Application error
Traceback (most recent call last):
File "grpcapp/.venv/lib/python3.12/site-packages/grpclib/server.py", line 446, in request_handler
await method_func(stream)
File "grpcapp/app/compiled_grpc2/test/msg/__init__.py", line 311, in __rpc_subscribe
await self._call_rpc_handler_server_stream(
File "grpcapp/.venv/lib/python3.12/site-packages/betterproto/grpc/grpclib_server.py", line 31, in _call_rpc_handler_server_stream
await stream.send_message(response_message)
File "grpcapp/.venv/lib/python3.12/site-packages/grpclib/server.py", line 196, in send_message
await send_message(self._stream, self._codec, message, self._send_type)
File "grpcapp/.venv/lib/python3.12/site-packages/grpclib/stream.py", line 44, in send_message
reply_bin = codec.encode(message, message_type)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "grpcapp/.venv/lib/python3.12/site-packages/grpclib/encoding/proto.py", line 47, in encode
return message.SerializeToString()
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "grpcapp/.venv/lib/python3.12/site-packages/betterproto/__init__.py", line 1139, in SerializeToString
return bytes(self)
^^^^^^^^^^^
File "grpcapp/.venv/lib/python3.12/site-packages/betterproto/__init__.py", line 1027, in __bytes__
self.dump(stream)
File "grpcapp/.venv/lib/python3.12/site-packages/betterproto/__init__.py", line 978, in dump
_serialize_single(
File "grpcapp/.venv/lib/python3.12/site-packages/betterproto/__init__.py", line 465, in _serialize_single
value = _preprocess_single(proto_type, wraps, value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "grpcapp/.venv/lib/python3.12/site-packages/betterproto/__init__.py", line 416, in _preprocess_single
return bytes(value)
^^^^^^^^^^^^
TypeError: string argument without an encoding
