ApplicationRunner should support unix-domain sockets

[meejah]

Currently it's only possible to open TCP (or TLS) connections on IPv4. This should include other socket types that Autobahn supports.

We can't just export Twisted endpoints as "the" API as that can't easily be supported for the asyncio ApplicationRunner...

[oberstet]

Yes, that is a major issue.

FWIW, CB lets you connect your component over Unix domain sockets as well. It can do that, as it uses the raw classes of AutobahnPython, not the ApplicationRunner convenience thing https://github.com/crossbario/crossbar/blob/master/crossbar/worker/container.py#L281

Maybe we need to define a goal first ..

[hynek]

If someone could point me to code that connects to CB using unix domain sockets I’d be most grateful. :)

(and if there’s a way to use raw sockets I’d be even more delighted by any example :))

[oberstet]

@hynek currently, ApplicationRunner lacks the necessary bits. @meejah and I are working on this. At the moment, only CB is able to host components which connect over UDS (and RawSocket and MsgPack) back to a router.

Here is an example: https://github.com/crossbario/crossbarexamples/blob/master/rest/caller_performance/.crossbar/config_multi_worker.json

[meejah]

Work is continuing on https://github.com/tavendo/AutobahnPython/tree/refactor-transport and feedback is appreciated on the API; there are examples showing its use.

[meejah]

Since this refactoring also involves TLS (among other) configuration, @oberstet and I had discussed "why would I want to pass a Twisted-native or asyncio-native (i.e. ssl.SSLContext) object"? Some use-cases would be:

• self-signed certificates
• the (default) system-wide certificate store is unsuitable (for some reason)
• you have a client-side certificate to load + supply
• you wish to do certificate-pinning (and/or other custom cert-chain verification)
• you have specific requirements for cipher-selector (or other TLS options?)

[om26er]

Does the component API support connecting to a unix socket ? I am working on something that requires to connect to a unix socket created by crossbar.

[haizaar]

@hynek, I found some here if you are still interested 3 years after :)

@oberstet, any chance to graduate ApplicationRunnerRawSocket from examples into a mainline?

[oberstet]

maybe we should nail this;)

fwiw, AB has a UDS aware WebSocket URL parser:

• https://github.com/crossbario/autobahn-python/blob/941600534ea28120342b39ad53c71aa920abb54c/autobahn/websocket/util.py#L125

and CB has a client connecting twisted endpoint:

• https://github.com/crossbario/crossbar/blob/cfa1feb938a7409ebae103649d1b47de29226101/crossbar/common/twisted/endpoint.py#L581

[meejah]

@om26er yes, Component supports this AFAIK.

[meejah]

Okay, looking again at the code, for Twisted, you can pass endpoint= to use any kind of endpoint you like (including Unix Domain Sockets). There is no such thing on the asyncio side -- asyncio has no "endpoint"-like concept, and none of our code is set up to call loop.create_unix_connection

What might make the most sense here is to use the component code. Ideally, this would be that we'd pass "endpoint configuration" to the ApplicationRunner (similar to Component) and it uses the component code to build transports. That won't quite work, because we've sort of mixed up "transport" plus other options -- e.g. url is required, and there's realm=, serializers= and ssl= which are all "transport configuration" options...but also re-connection options, etc.

So, maybe we could introduce a transport_config= kwarg to both twisted + asyncio ApplicationRunners and (if it's provided) that is used to construct the transport to connect to (using the existing component code, like autobahn.wamp.component._create_transport() and the framework-specific _create_transport_factory calls. This would at least be less code duplication, and bring ApplicationRunner and Component closer together (so far as "how do I tell it where to connect" is).

[oberstet]

That won't quite work, because we've sort of mixed up "transport" plus other options

yes, unfortunately.

So, maybe we could introduce a transport_config= kwarg to both twisted + asyncio ApplicationRunners and (if it's provided) that is used to construct the transport to connect to

that sounds good to me!

IMO, a good goal would be: it should be able to run from the ~same config as

def create_connecting_endpoint_from_config(config, cbdir, reactor, log):
    """
    Create a Twisted stream client endpoint from a Crossbar.io transport configuration.

in CB. we could move that code from CB to AB (no one touched it but us, so no license issues) so further reduce code duplication