'TypeError: buf must be a byte string' using Python 2

[asmeurer]

In Python 2, I keep getting

  File "/Users/aaronmeurer/anaconda/envs/sympy-release/lib/python2.7/site-packages/requests/api.py", line 94, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/Users/aaronmeurer/anaconda/envs/sympy-release/lib/python2.7/site-packages/requests/api.py", line 49, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/aaronmeurer/anaconda/envs/sympy-release/lib/python2.7/site-packages/requests/sessions.py", line 457, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/aaronmeurer/anaconda/envs/sympy-release/lib/python2.7/site-packages/requests/sessions.py", line 569, in send
    r = adapter.send(request, **kwargs)
  File "/Users/aaronmeurer/anaconda/envs/sympy-release/lib/python2.7/site-packages/requests/adapters.py", line 362, in send
    timeout=timeout
  File "/Users/aaronmeurer/anaconda/envs/sympy-release/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 516, in urlopen
    body=body, headers=headers)
  File "/Users/aaronmeurer/anaconda/envs/sympy-release/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 308, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/Users/aaronmeurer/anaconda/envs/sympy-release/lib/python2.7/httplib.py", line 995, in request
    self._send_request(method, url, body, headers)
  File "/Users/aaronmeurer/anaconda/envs/sympy-release/lib/python2.7/httplib.py", line 1029, in _send_request
    self.endheaders(body)
  File "/Users/aaronmeurer/anaconda/envs/sympy-release/lib/python2.7/httplib.py", line 991, in endheaders
    self._send_output(message_body)
  File "/Users/aaronmeurer/anaconda/envs/sympy-release/lib/python2.7/httplib.py", line 844, in _send_output
    self.send(msg)
  File "/Users/aaronmeurer/anaconda/envs/sympy-release/lib/python2.7/httplib.py", line 820, in send
    self.sock.sendall(data)
  File "/Users/aaronmeurer/anaconda/envs/sympy-release/lib/python2.7/site-packages/requests/packages/urllib3/contrib/pyopenssl.py", line 203, in sendall
    return self.connection.sendall(data)
  File "/Users/aaronmeurer/anaconda/envs/sympy-release/lib/python2.7/site-packages/OpenSSL/SSL.py", line 970, in sendall
    raise TypeError("buf must be a byte string")
TypeError: buf must be a byte string

I debugged, and I am pretty sure it is coming from requests-oauthlib, which converts the body into a unicode string somewhere when it does its thing with the OAuth2 token (it could also be oauthlib, I haven't debugged that much). The only way I could work around it was to edit that SSL.py and make it encode the buf instead of raising an exception (I would use Python 3 except I have a hard dependency on fabric which doesn't support it yet).

I looked and requests-oauthlib and oauthlib are both making heavy use of from __future__ import unicode_literals, which if I had to guess, is probably the source of the problem.

[ib-lundgren]

Python 2 is fine and if its caused by requests oauthlib or oauthlib we will fix it.

Not sure I can reproduce from the trace alone, can you please include instructions for reproducing and/or logging output for both the 'oauthlib' and 'requests-oauthlib' namespace. Cheers!

[asmeurer]

This is the code that did it https://github.com/sympy/sympy/blob/8cd27ddecec7f275f20b0d4d35ba16071eb5216b/release/fabfile.py#L1269

[avidas]

+1 ran into the same issue on Python 2.6

[asmeurer]

Removing from __future__ import unicode_literals from the code will likely fix this. Its use is typically a bad idea (unlike the other __future__ imports) because not everything in Python 2 "just works" if passed a unicode string instead of a str (as this error shows).

[cosimo]

Had the same problem with Python 2.7 on Ubuntu 14.04 LTS. I tried removing the from __future__ import unicode_literals statements but that didn't really work for me.

[sigmavirus24]

@cosimo as @ib-lundgren pointed out. It's difficult to reproduce this by traceback alone. Can you provide more context to help repro this?

[cosimo]

I can start from the SSL error and trace my steps back, but I have no clear idea of what might be the underlying issue.