No way to detect "dead blocks"

[helmus]

Since there is no way to guarantee that an async api is going to play nice and invoke it's callback, suspend could provide a way to govern this. In my current node code that uses an caolan/async variant i use a time out guard for each async operation that will invoke an exception with the relevant call stack if the async operation does not return in a timely fashion. This technique has been pretty useful for me during debugging and also adds in detecting slow api calls.

Do you think this sort of time out functionality could be useful in suspend ? Maybe you have any other ideas or suggestion on how we could provide some sort of more graceful fallback when a generator "dead blocks" ?

I will prepare a PR when i have some more time in the next few days.

[spion]

You could always do the following:

yield operation(arg, timeout(resume, 2000));

where:

function timeout(resume, t) {
  var complete = false;
  var timer = setTimeout(function() {
    if (complete) return;
    complete = true;
    resume(new TimeoutError("The operation has timed out after " + t + "ms"));
  }, t);
  return function timeoutCallback() {
    if (complete) return;
    clearTimeout(timer);
    complete = true; 
    resume.apply(this, arguments);
  }
}

No need to add more features to the resumer.

Or did you mean setting a global timeout for all operations?

[helmus]

That's a nice way of handling this problem in an explicit way. But this implies that you know when certain api calls are suspect to bad behavior, or you have to wrap all calls with this timeout function. The point of governing the generator is to make sure you don't have to think when a function might timeout.

@spion , A default per generator setting is what i had in mind, configuring it per operation seems useful to, but this should then be part of the api someway to be consistent with a per generator setting.

I think default would be useful, because currently if a generator "dead-blocks" there is no call stack or anything available to the developer.

[jmar777]

@helmus In the past I've handled situations like that with https://github.com/jmar777/cb. I'll stew over this some more, but my initial thoughts are that this may be outside the scope of what suspend (in its current state) should be concerned with.

I say "current state", though, because I am trying to play with some alternative API's for suspend that would make it easier to opt in to more robust behavior (while remaining minimal in its "default" form). This is going to get off topic quickly, but I'd love to get feedback from you or @spion on this. Here's my own current list of issues with suspend that I think dictate a bit of an overhaul (that perhaps could include timeout behavior). The goal would be to get these in before (or soon after) Node v0.12.0, at which point I think suspend's API should start stabilizing:

---

Problem: Better messages for evil callbacks (invoked multiple times). Right now they just receive a very vague error message. Possible Solution: Perhaps resume should work more like genny, and require being invoked: resume().

---

Problem: The resume parameter was previously made to be the last parameter, with the intention of making it optional when using Promises. This turns out to be painful in some situations, though. For example, when a lot of other parameters are present:

[].forEach(suspend(function* (el, idx, arr, resume) {
    // note, this is just a contrived example and not really a recommended way to do
    // iteration in suspend...
})

Possible Solution: Move resume back to first arg, but make it truly optional. Maybe this implies introducing an options object for this:

suspend(function* (resume) {
    ...
}, { resume: true });

At that point the question would be what the default should be (true|false). That's also a bit ugly, so an alternative is to stick with the chaining syntax used elsewhere (suspend(...).noResume()). Maybe all options are provided through an options object and the chaining API...?

Related question: now that Function#length is spec'd to be configurable, it would be nice to preserve arity in the wrapped function, but do we include resume in that arity? Maybe that's another option.

---

Problem: There's no way to expose a suspend-wrapped generator for delegation (via yield*). Possible Solution: A new delegate option? With the chaining syntax, something like:

suspend(function* () {
    yield* suspend(function* () {
        ...
    }).delegate();
});

---

Problem: No support for parallel operations (other than wrapping inside of other async control-flow utilities) or other higher-level control-flow constructs. Possible Solution: Perhaps we need suspend.parallel, suspend.waterfall, etc. (a la async). The current suspend(fn*) syntax could be an alias for a suspend.run(fn*) utility method.

---

Problem: The combination of Promises and generators is pretty slick, and with Promises making their way into ES6 it would potentially be nice to opt in to having suspend transparently resolve to a promise. Along the same lines, if there was a way to indicate that one of the parameters was an error-first callback, we could pass thrown exceptions or return values to it. Possible Solution: Not really sure on this one right now... kind of depends on how the rest of the API shapes up.

---

</brain-puke>

Anyway, that's quite a lot and definitely beyond the scope of this particular issue. I wanted to throw this out there because you both have a good grasp on suspend and seem to have an interest in its future (which I am very grateful for). I've rewritten suspend about 4 different ways now trying to accommodate all this, but I keep running into the usual conflicts between "simple vs easy", "terse vs. robust", etc.

I'm working on sketching out some API options on this right now, but any and all feedback or suggestions would be greatly appreciated. :)

[spion]

Resume parameter

Looks like why solves this by avoiding having a resume argument alltogether. Not sure if thats doable without taking over this - need to have a closer look. Perhaps it has a currentGenerator global?

Calling unwrapped generators

galaxy has unstar, and genny has resume.gen() which is analogous to resume but for generators. why's method is probably the first avenue to explore...

Parallel operations

genny lets you postpone yielding - not sure if its the best solution, I think it works pretty well but it could be made to look just a bit less magical perhaps?

Promises

Yeah, awesomeness! I suppose the wrapped function could support a dual promise-callback style API which is slowly becoming common practice?

[helmus]

generator delegation

For readability i think it would be cooler if the delegate function would follow right after the yield* statement. Not to sure on a useful name or anything though, but it should definitely be possible.

suspend(function* () {
    yield* suspend.To(someSuspendFunction);
});

Promises / Parallel

After taking a closer look at bluebird I must say i really do feel that i feel that promises solves this in a very elegant way.

It solves the resume option ( you can just omit it al together ), and it makes it really easy to parallelize without having to do bind anything or think to hard.

yield Promise.all([   
   doParalell1(),
   doParalell2(),
   doParalell3()
]);

You could even do some pretty advanced delayed parallelisation with some pretty simple syntax like this:

var x = 0, parallel = [];
for (; x < 5; x++){
    parallel .push(doEventually(x));
}
yield Promise.all(parallel);

The biggest drawback being of course that you have to wrap everything and everything. So i'm still not sure how i feel about that. I must say though that i also like parallel approach that genny takes.

Evil callbacks

I'm somewhat confused on this. If a library has multiple callbacks without signaling an end then it's probably an event and not really an async function ( right ? ). So including it in a suspend "thread" ( if you will ) wouldn't seem like the right thing to do. If the multiple callbacks are actually a bug, then i don't think suspend should support using them explicitly. I think some more concrete use cases could increase my understanding on this.

[jmar777]

Thanks for the feedback, guys. @spion - checking out why now. There's some dark magic going on in there... cool syntax, but I think understanding the implementation is going to take another pass or two lol.

[jmar777]

Hey @spion and @helmus - I've been giving some more thought towards the API. I'm posting it here both for the sake of having some reference notes, and to also solicit any feedback. Your feedback has been hugely helpful, and I of course would love to hear any further thoughts on this.

Here's what I'm currently thinking for the v0.4.0 release, with the goal of making this the last major backwards-incompatible release before node v0.12.0 hits. For the promise-based examples, assume var readFile = Q.denodeify(fs.readFile);.

suspend.async(fn*)

Creates a new suspend-wrapped generator function. Functionally equivalent to the current behavior of suspend(function* () {}).

var run = suspend.async(function* () {
    var data = yield readFile('/tmp/foo', 'utf8');
});
run();

Side note: is there a better name than async() here? Not totally thrilled with it, but it's the best I can come up with.

suspend(fn*)

API sugar for suspend.async(fn*). Maintains compatibility with current API, other than the resume parameter.

var run = suspend(function* () {
    var data = yield readFile('/tmp/foo', 'utf8');
});
run();

suspend.run(fn*)

Runs immediately. Functionally equivalent to suspend(fn*)().

suspend.run(function* () {
    var data = yield readFile('/tmp/foo', 'utf8');
});

suspend.resume()

"Resumer" factory, operates similar to Y().

suspend(function* () {
    var data = yield fs.readFile('/tmp/foo', 'utf8', suspend.resume());
});

suspend.fork() and suspend.join()

Primitives for parallel operations.

Basic example:

suspend(function* () {
    fs.readFile('/tmp/foo', 'utf8', suspend.fork());
    fs.readFile('/tmp/bar', 'utf8', suspend.fork());
    var results = yield suspend.join();
    // results == ['foo contents', 'bar contents']
});

suspend.join([promise1, promise2, ..., promiseN])

Allows parallel operations using Promises.

suspend(function* () {
    var results = yield suspend.join([
        readFile('/tmp/foo', 'utf8'),
        readFile('/tmp/bar', 'utf8')
    ]);
    // results == ['foo contents', 'bar contents']
});

suspend.fork(key) and suspend.join()

Allows map-like behavior with keyed data.

suspend(function* () {
    fs.readFile('/tmp/foo', 'utf8', suspend.fork('foo'));
    fs.readFile('/tmp/bar', 'utf8', suspend.fork('bar'));
    var results = yield suspend.join();
    // results == { foo: 'foo contents', bar: 'bar contents' }
});

suspend.join({})

Allows keyed mapping using promises.

suspend(function* () {
    var results = yield suspend.join({
        foo: readFile('/tmp/foo', 'utf8'),
        bar: readFile('/tmp/bar', 'utf8')
    });
    // results == { foo: 'foo contents', bar: 'bar contents' }
});

As with any API, there obviously some pros and cons here, but on the whole I'm fairly happy with it. I think it provides a natural path forward for additional functionality, such as suspend.delegate() and other API niceties. Thanks again!

[spion]

I like the API, but I believe the most common action - creating a resumer - should be shorter. Have you tried it on some sample code?

OTOH you're right, the user could import suspend like so:

var async = require('suspend').async, resume = require('suspend').resume;

async(function* () {
  var x = yield fn('arg', resume());
  //...
});