boto3 icon indicating copy to clipboard operation
boto3 copied to clipboard

DynamoDB's Binary type issues with Python 3

Open miguelgrinberg opened this issue 9 years ago • 7 comments

The Binary class has a problematic implementation of the __str__() method that assumes str and bytes are interchangeable. Example:

Python 3.5.2 (default, Sep 28 2016, 18:11:34)
>>> from boto3.dynamodb.types import Binary
>>> b = Binary(b'hello')

>>> b
Binary(b'hello')

>>> print(b)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __str__ returned non-string (type bytes)

>>> bytes(b)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'Binary' object is not iterable

The above examples all succeed under Python 2.7.

miguelgrinberg avatar Oct 12 '16 22:10 miguelgrinberg

Makes sense to have. We will take a look at your PR as well to make sure this gets implemented.

kyleknap avatar Oct 14 '16 21:10 kyleknap

Bumping this issue as I just encountered it while using python3. Any idea if the fix presented by @miguelgrinberg is acceptable?

ghmj2417 avatar Jul 13 '18 15:07 ghmj2417

Why is this marked as feature-request? It's a bug that happens in Python3.

Also, why is the Binary wrapper class needed at all?

scribu avatar Sep 20 '18 10:09 scribu

This is currently preventing us from properly utilizing the Binary field in DynamoDB in our application. Is there an ETA on a potential fix for this issue?

thetestgame avatar Oct 26 '18 03:10 thetestgame

FYI: I submitted a PR that addresses this issue (more than two years ago!!!): https://github.com/boto/boto3/pull/848. So if your process allows it, you can install the PR branch to avoid the issue. Unfortunately the maintainers are slow or not interested in getting this fix out. :(

miguelgrinberg avatar Oct 26 '18 10:10 miguelgrinberg

I encountered this issue also. Found I could work around it by accessing the value directly. E.g.

Python 3.5.2 (default, Nov 12 2018, 13:43:14)
>>> from boto3.dynamodb.types import Binary
>>> b = Binary(b'hello')
>>> b
Binary(b'hello')

>>> print(b.value)
b'hello'

>>> print(b.value.decode("utf-8"))
hello

mhball avatar Jun 21 '19 15:06 mhball

Just wanted to check in here and provide some updates. This seems like Python 2 vs Python 3 issue and can potentially be fixed but we would need to do more investigation to find backwards-compatible solution. This is currently a low priority for us but please feel free to check in back here for updates.

For now, I would refer to the workaround provided by @mhball in previous comment. Thanks for sharing the workaround with others ❤️

aBurmeseDev avatar Dec 15 '22 03:12 aBurmeseDev