Trying to iterate recursively, and use blocking operations arg

[cyisfor]

trafficstars

I'm trying to do a thing that iterates through a tree, possibly a big tree, possibly one I wouldn't want to load all into memory at once, and it might be partially on the cloud so I wanted to pause in places to wait for items to come in.

So I can figure out how to iterate recursively using yield, and how to use bluelet to block on events using yield, but not to do both!

I tried to illustrate what I'm talking about with a program, attached...

[cyisfor]

import bluelet

# Let's say there's a recursive generator...
# Either it generates a ValueEvent, or it delegates to the next level of recursion
# Shouldn't that ...sometimes... yield value events?

def justIteration():
    def listgetter(depth,i):
        if depth == 0:
            yield i
            return
        accum = 0
        for i in range(3):
            for thing in listgetter(depth-1,i):
                accum += thing
                yield accum
    for thing in listgetter(3,4):
        print(thing)
justIteration()

def justBlocking():
    def listgetter(depth,i):
        # some blocking socket operation here maybe.
        print("blocking yay")
        yield bluelet.sleep(1)
        yield bluelet.end(42)
    def main():
        value = yield listgetter(3,4)
        print(value)
    bluelet.run(main())

justBlocking()

def both():
    def listgetter(depth,i):
        if depth == 0:
            yield bluelet.end(i)
            return
        accum = 0
        for i in range(3):
            item = yield bluelet.call(listgetter(depth-1,i))
            # some blocking operation here.
            yield bluelet.sleep(0.1)
            accum += item
            print('Yielding a value event here!',accum)
            yield bluelet.ValueEvent(accum)
        yield bluelet.end(42)

    def main():
        for event in listgetter(3,4):
            if type(event)==bluelet.ValueEvent:
                # this is never called. Where are the value events?
                print('A value event was yielded.',event.value)
            else:
                # re-yield event?
                print('delegating')
                yield event

    bluelet.run(main())
both()

[cyisfor]

OK I think this might be doing it. I don't even comprehend what I'm doing at this point. Have to look and see what the heck I did.

def both():
    def listgetter(depth,i):
        if depth == 0:
            yield bluelet.end(i)
            return
        accum = 0
        for i in range(3):
            for item in listgetter(depth-1,i):
                # some blocking operation here.
                if hasattr(item,'value'):
                    item = item.value
                    print(item)
                    yield bluelet.sleep(0.1)
                    accum += item
                else:
                    yield item
            print('Yielding a value event here!',depth,accum)
            yield bluelet.ValueEvent(accum)
        yield bluelet.end(accum)

    def main():
        for item in listgetter(3,4):
            if hasattr(item,'value'):
                print('A value was yielded.',item.value)
            else:
                yield item
    bluelet.run(main())
both()

[sampsyo]

This is an interesting case—I haven't really built support for (nor thought of a good idea for how to support) using Bluelet within generators (coroutines) that you actually intend to use as generators. That is, Bluelet assumes that you're just using yield to generate events and that the Bluelet scheduler will have full control over them.

So, while I don't really know what the best way to combine iteration with Bluelet events is, your solution is actually a pretty neat hack. main passes most through events from listgetter to the scheduler, but intercepts certain events and consumes them before sending them. Nice idea—I don't really have a better one for now, so :+1: to this from me.