Python 3.6/Raven 6.0.0 - Twisted transport incompatible with dsn passed to Client

[mike-hart]

More code required below for a twisted eventloop, sorry, this is ripped from a larger project.

client = Client(dsn=some_value, transport=TwistedHTTPTransport)
client.captureMessage(message='hello')

if the DSN is a string then we get:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/raven/base.py", line 708, in send_remote
    failed_send)
  File "/usr/lib/python3.6/site-packages/raven/transport/twisted.py", line 45, in async_send
    headers=Headers(dict((k, [v]) for k, v in headers.items()))
  File "/usr/lib/python3.6/site-packages/twisted/web/client.py", line 1623, in request 
    parsedURI = URI.fromBytes(uri)
  File "/usr/lib/python3.6/site-packages/twisted/web/client.py", line 630, in fromBytes
    scheme, netloc, path, params, query, fragment = http.urlparse(uri)
  File "/usr/lib/python3.6/site-packages/twisted/web/http.py", line 180, in urlparse 
    raise TypeError("url must be bytes, not unicode")
TypeError: url must be bytes, not unicode

but if the DSN is bytes then we have:

Unhandled Error
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/twisted/python/usage.py", line 447, in __str__
    return self.getSynopsis() + '\n' + self.getUsage(width=None)
  File "/usr/lib/python3.6/site-packages/twisted/python/usage.py", line 483, in getUsage 
    for (cmd, short, parser, desc) in self.subCommands:
  File "/usr/lib/python3.6/site-packages/twisted/application/app.py", line 641, in subCommands
    for plug in sorted(plugins, key=attrgetter('tapname')):
  File "/usr/lib/python3.6/site-packages/twisted/plugin.py", line 213, in getPlugins
    allDropins = getCache(package)
--- <exception caught here> --- 
  File "/usr/lib/python3.6/site-packages/twisted/plugin.py", line 171, in getCache 
    provider = pluginModule.load()
  File "/usr/lib/python3.6/site-packages/twisted/python/modules.py", line 392, in load
    return self.pathEntry.pythonPath.moduleLoader(self.name)
  File "/usr/lib/python3.6/site-packages/twisted/python/reflect.py", line 301, in namedAny 
    topLevelPackage = _importAndCheckStack(trialname)
  File "/usr/lib/python3.6/site-packages/twisted/python/reflect.py", line 240, in _importAndCheckStack
    return __import__(importName)
  File "/opt/myplugin/twisted/plugins/myplugin.py", line 31, in <module>
    myplugin.utils.setup_sentry()
  File "/opt/myplugin/something/utils.py", line 47, in setup_sentry
    dsn=settings.SENTRY_DSN, transport=TwistedHTTPTransport
  File "/usr/lib/python3.6/site-packages/raven/base.py", line 167, in __init__ 
    self.set_dsn(dsn, transport)
  File "/usr/lib/python3.6/site-packages/raven/base.py", line 248, in set_dsn
    transport_registry=self._registry,
  File "/usr/lib/python3.6/site-packages/raven/conf/remote.py", line 102, in from_string
    netloc += ':%s' % url.port 
builtins.TypeError: can't concat bytes to str

[990px]

EDIT: works with version < 6.x

Quick workaround to this bug:

from raven.transport.twisted import TwistedHTTPTransport

class FixedTwistedHTTPTransport(TwistedHTTPTransport):
    def __init__(self, parsed_url, *args, **kwargs):
        super().__init__(parsed_url, *args, **kwargs)

        # twisted wants bytes
        if isinstance(self._url, str):
            self._url = str.encode(self._url)


client = Client(dsn=RAVEN_DSN, transport=FixedTwistedHTTPTransport)

[mike-hart]

Unless I'm missing something very important, self._url is not something used anywhere other than the tests. This change results in a hang as the attempt to log an error results in an error which results in an attempt to log an error, which results in... etc. This causes the process to hang for an extended period at 100% CPU. The only way I found to fix this was to do the encode of the url inside the send_async method, basically copying the bulk of the implementation.

[990px]

You are right. I didn't notice that you are talking about 6.x version. I'm using v5.x. The only way is to overload send_async or Agent.request.

[tonal]

Fix for raven==6.5.0:

    class FixTwistedHTTPTransport(TwistedHTTPTransport):
        url_encode = 'utf-8'
        def async_send(self, url, data, headers, success_cb, failure_cb):
            b_url = url.encode(url_encode)
            return super().async_send(b_url, data, headers, success_cb, failure_cb)

    client = raven.Client(dsn=dsn, transport=FixTwistedHTTPTransport)