typeshed icon indicating copy to clipboard operation
typeshed copied to clipboard

`urllib.request.Request.data` type erroring when using `decode`

Open arkinmodi opened this issue 1 year ago • 2 comments

When getting the data out of a urllib.request.Request object, mypy is erroring saying that decode is not supported even tho it is a bytes object (and does not error when at runtime).

$ cat t.py
from urllib.request import Request

req = Request("http://localhost:8080", data=b"req body")

if req.data:
    print(dir(req.data))
    print(type(req.data))
    print(req.data.decode())

$ mypy t.py
t.py:8: error: Item "Buffer" of "Buffer | SupportsRead[bytes] | Iterable[bytes]" has no attribute "decode"  [union-attr]
t.py:8: error: Item "SupportsRead[bytes]" of "Buffer | SupportsRead[bytes] | Iterable[bytes]" has no attribute "decode"  [union-attr]
t.py:8: error: Item "Iterable[bytes]" of "Buffer | SupportsRead[bytes] | Iterable[bytes]" has no attribute "decode"  [union-attr]
Found 3 errors in 1 file (checked 1 source file)

$ python t.py
[... 'decode', ...]
<class 'bytes'>
req body

$ python --version --version
Python 3.12.0 (main, Oct  2 2023, 20:56:14) [Clang 16.0.3 ]

$ mypy --version
mypy 1.8.0 (compiled: yes)

I believe PR #10225 had made this an error as it changed bytes to Buffer.

- ReadOnlyBuffer: TypeAlias = bytes
- ReadableBuffer: TypeAlias = ReadOnlyBuffer | WriteableBuffer  # stable
+ ReadOnlyBuffer: TypeAlias = Buffer  # stable
+ ReadableBuffer: TypeAlias = Buffer  # stable

Before: https://github.com/python/typeshed/blame/56aeeb677f27c6771e683314621521aff5c97cd1/stdlib/_typeshed/init.pyi#L230-L241 After: https://github.com/python/typeshed/blame/1e7e174b4fc4da6a66622168f8012c7ee0499392/stdlib/_typeshed/init.pyi#L263-L267


Deferred until #11422.

arkinmodi avatar Feb 27 '24 00:02 arkinmodi

Isn't it because the data property can also be Iterable[bytes] so .decode() is not available? I think that's not your case, but it's a possible case.

aminalaee avatar Feb 28 '24 13:02 aminalaee

The problem here is that the type of the data attribute depends on the data passed in, which might not have a decode() method. The solution is to make Request generic over the type of data, but we'll have to wait until #11422 is implemented. I've marked this issue as "deferred" for now.

srittau avatar Mar 15 '24 14:03 srittau