True single source support for app developers

[oberstet]

txaio - as it is - is a low-level helper library that allows other library implementers to write single-source libraries that work unmodified across Twisted and asyncio. Like for example autobahn. One needs to follow a certain programming style too to have a single-source, also over different Python versions. For library implementors, I'd say this is a fair trade-off between effort and reach.

Now, for application developers on the other hand, this is probably too much burden, and application developers might prefer less effort, and more comfort and accept a requirement like Python 3.5+ for their application - mainly because applications can be bundled in a more or less self-contained way that includes whatever Python is required.

one needs to use Python 3.5+ and latest Twisted with async/await support to use co-routines for true single-source (the alternative is to use pure futures/deferreds style - which then can support Python 2.7+)

I think we are nearly there. Just a little more glue, pastering over gaps, and it should be possible to write true single source applications that work on Twisted and async unmodified.

For example, it would be nice to have a simple and robust way of starting a async program written using txaio like this:

async def main(reactor_or_loop):
    # create an etcd client
    etcd = Client(reactor_or_loop, u'http://localhost:2379')

    # retrieve etcd cluster status
    status = await etcd.status()
    print(status)

if __name__ == '__main__':
    import txaio
    if argv[1] == 'asyncio':
        txaio.use_asyncio()
    elif argv[1] == 'twisted':
       txaio.use_twisted()
    else:
        raise Exception('must chose between "twisted" and "asyncio"')
    txaio.start_logging(level='info')
    txaio.run(main)  # works like Twisted react, also on asyncio

This code is then supposed to run on either Twisted or asyncio - unmodified.

The example is adapted from the txaio-etcd library (which would need support for asyncio under the hood, but this can be done).

Maybe there are more gaps etc. But this is the idea: truly fulfil the original promise of txaio.

Well, maybe it's limited interest, because why would an application developer not just select on one networking framework beforehand and early on, and then write code that is specific to that framework?

Anyway, at least I find the idea compelling;)

What do you think?

[oberstet]

Autobahn follows these rules (in it's cross network framework parts):

• only plain Deferreds/Futures, no co-routines
• everything managed via txaio.create_future etc
• using six and strict bytes/unicode (b".. vs u"..")

And thus can support anything from Python 2.7 upwards with both Twisted and asyncio (the latter being polyfilled for Python <3.3).

Crossbar.io has these rules:

• Twisted only
• Python 2.7 is supported, and hence no await/async, but Twisted inlineCallbacks

We do want Twisted here because it has a much broader scope than asyncio, and includes support for a lot things already. So requiring Python 3.5+ would bring as a minor syntactical win. I have no idea about performance if using await/async vs the Twisted "hack" of using exceptions for co-routines (which is ingenius really). But PyPy isn't at 3.5 anyway - and this is a killer.

[meejah]

I'm not 100% sure, but I think the "async def" and "await" is mostly-just syntax sugar on top of the slightly-older python3 thing of @coroutine and yield from so I wouldn't expect any major performance difference -- but I'd want to see that tested/proved before wholesale switching Crossbar to 3.5+ syntax when PyPy supports that.