Implement __await__ on operation classes

[sjlongland]

So, been thinking about this for a long while now. We have the necessary bits for asynchronous operations, and yes, it's on my to-do list to write an async HTTP client implementation. Modern ipython (and probably Jupyter notebook too) supports the await keyword.

It'd be real nice to be able to run:

res = await client.his_read_frame(…etc…)

Turns out, this can be done, with a little work on our end: https://docs.python.org/3.5/library/collections.abc.html#collections.abc.Awaitable

We'd need to implement (in Python 3.5+ only):

class HaystackOperation(object):
    if CAN_PYTHON_AWAIT:
        def __await__(self):
            future = Future()
            def _on_done(*args, **kwargs):
                try:
                    future.set_result(self.result)
                except Exception as ex:
                    future.set_exception(ex)
            self.done_sig.connect(_on_done)
            return future

This is untested of course, and we'd need to write some code to detect if we're on Python 3.5+ since this won't work on anything older. That has the potential though to make a much nicer experience for users, particularly interactive users.

[sjlongland]

Ohhh so much nicer!

In [3]: points = await session.find_entity('point and cmd and wshHubRef->wshEui64=="00124b0018e4237f"')
DEBUG:urllib3.connectionpool:Resetting dropped connection: ….on.widesky.cloud
DEBUG:urllib3.connectionpool:https://….on.widesky.cloud:443 "GET /widesky/api/read?filter=point+and+cmd+and+wshHubRef-%3EwshEui64%3D%3D%2200124b0018e4237f%22 HTTP/1.1" 200 None
DEBUG:pyhaystack.client.WideskyHaystackSession.find_entity:Received grid: <Grid>
        Version: 2.0
        Columns:
                id
                cmd
                cur
                dis
                equipRef
                fqname
                kind
                name
                point
                siteRef
                writable
                writeLevel
                writeStatus
                writeVal
                wsgRealTimeChangeOfState
                wsgRealTimeInterval
                wsgRealTimeOffset
                wshHubRef
                wshPoint
        Row    0:
        id=Ref('926f9ad8-0585-4cd0-b4ea-90232609c7e9', 'Control point', True)
        cmd=MARKER
        cur=MARKER
        dis='Control point'
        equipRef=Ref('99c689cc-66d4-4db6-b159-25ef0699578a', '00124b0018e4237f H-Bridge on port 1', True)
        fqname='ws.hub237fhb1.ctl'
        kind='Bool'
        name='ctl'
        point=MARKER
        siteRef=Ref('70624619-de61-4f6c-aaf7-014443123f39', 'WideSky Hub "Test Site"', True)
        writable=MARKER
        writeLevel=17.0
        writeStatus='ok'
        writeVal=True
        wsgRealTimeChangeOfState=60.0
        wsgRealTimeInterval=60.0
        wsgRealTimeOffset=0.0
        wshHubRef=Ref('ba0a5a46-cef9-4d4f-841e-0fcb99e192cd', 'Hub 00124b0018e4237f (@The Gap)', True)
        wshPoint=16.0
        Row    1:
        id=Ref('4b93cbba-db3d-418c-a229-322016ef30ef', 'Control point', True)
        cmd=MARKER
        cur=MARKER
        dis='Control point'
        equipRef=Ref('61b67447-cff3-421b-8e87-7952918bf996', '00124b0018e4237f H-Bridge on port 2', True)
        fqname='ws.hub237fhb2.ctl'
        kind='Bool'
        name='ctl'
        point=MARKER
        siteRef=Ref('70624619-de61-4f6c-aaf7-014443123f39', 'WideSky Hub "Test Site"', True)
        writable=MARKER
        writeLevel=17.0
        writeStatus='ok'
        writeVal=True
        wsgRealTimeChangeOfState=60.0
        wsgRealTimeInterval=60.0
        wsgRealTimeOffset=0.0
        wshHubRef=Ref('ba0a5a46-cef9-4d4f-841e-0fcb99e192cd', 'Hub 00124b0018e4237f (@The Gap)', True)
        wshPoint=14.0
        Row    2:
        id=Ref('d3fd360a-81e0-4235-a2f2-dcb423fbebb5', 'Control point', True)
        cmd=MARKER
        cur=MARKER
        dis='Control point'
        equipRef=Ref('7f4e8800-8294-4e05-93e5-4b3646ad378f', '00124b0018e4237f Relay on port 1', True)
        fqname='ws.hub237frl1.ctl'
        kind='Bool'
        name='ctl'
        point=MARKER
        siteRef=Ref('70624619-de61-4f6c-aaf7-014443123f39', 'WideSky Hub "Test Site"', True)
        writable=MARKER
        writeLevel=17.0
        writeStatus='ok'
        writeVal=True
        wsgRealTimeChangeOfState=60.0
        wsgRealTimeInterval=60.0
        wsgRealTimeOffset=0.0
        wshHubRef=Ref('ba0a5a46-cef9-4d4f-841e-0fcb99e192cd', 'Hub 00124b0018e4237f (@The Gap)', True)
        wshPoint=17.0
        Row    3:
        id=Ref('6b05ac19-f649-41e5-8607-73f7140da841', 'Control point', True)
        cmd=MARKER
        cur=MARKER
        dis='Control point'
        equipRef=Ref('f82990c2-f3a3-41be-aef3-b1fc6f834e89', '00124b0018e4237f Relay on port 2', True)
        fqname='ws.hub237frl2.ctl'
        kind='Bool'
        name='ctl'
        point=MARKER
        siteRef=Ref('70624619-de61-4f6c-aaf7-014443123f39', 'WideSky Hub "Test Site"', True)
        writable=MARKER
        writeLevel=17.0
        writeStatus='ok'
        writeVal=True
        wsgRealTimeChangeOfState=60.0
        wsgRealTimeInterval=60.0
        wsgRealTimeOffset=0.0
        wshHubRef=Ref('ba0a5a46-cef9-4d4f-841e-0fcb99e192cd', 'Hub 00124b0018e4237f (@The Gap)', True)
        wshPoint=15.0
</Grid>

Now, an asynchronous HTTP client and we're all set.

[ChristianTremblay]

Pushing me down the canyon of async/await... :-)

[sjlongland]

On 25/11/20 2:48 am, Christian Tremblay wrote:

Pushing me down the canyon of async/await... :-)

Well… it's got to be a more natural solution than:

op = doSomething() op.wait() res = op.result

Next step then would be to implement an asynchronous HTTP client. I'd be interested to see if await works in newer versions of Jupyter Notebook. Support is in ipython v6 and later.

There's also support for Tornado, so Python 2.7 users who want async action can yield op.future to do the same thing with Tornado v4.

[ChristianTremblay]

Joking. It's awesome. Really.

But I haven't yet be able to wrap my head around async... I'll need some help ;-)

[ChristianTremblay]

I'll keep this open and tag myself as documentation would probably be a good way for me to learn.... If I can find some time.