return doesn't work in for loop comprehensions

[edemaine]

Feature request (or maybe bug)

Input Code

```coffee
f = ->
  for x in [1..5]
    return null if bad x
    x

try link

Expected Behavior

I expected this code to return an array if none of the bad calls came up true:

var f;

f = function() {
  var i, results, x;
  results = [];
  for (x = i = 1; i <= 5; x = ++i) {
    if (bad(x)) {
      return null;
    }
    results.push(x);
  }
  return results;
};

Current Behavior

However, in the code generated by CoffeeScript, the function returns nothing if none of the bad calls come up true:

var f;

f = function() {
  var i, x;
  for (x = i = 1; i <= 5; x = ++i) {
    if (bad(x)) {
      return null;
    }
    x;
  }
};

Possible Solution

I'm guessing that this issue is caused by class Return declaring isStatement: YES, which makes sense by itself. But perhaps it could be worked around in "the hairiest method in all of CoffeeScript.", For::compileNode.

Context

I guess the main discussion to have is whether the existing behavior is intended. The documentation expresses the spirit of Everything is an Expression (at least, as much as possible), and further encourages:

Even though functions will always return their final value, it’s both possible and encouraged to return early from a function body writing out the explicit return (return value), when you know that you’re done.

So my feeling is that my expected behavior is "more correct", though it would technically be a backward-incompatible behavior. (Code that didn't return anything before would now return something.)

Environment

• CoffeeScript version: web try / 2.5.1

[jashkenas]

Hi Erik.

Great question! The current behavior is intentional — although mixing statement-oriented control flow code, and expression-oriented value code is certainly a bit of a wrinkle.

By placing a return statement within a loop (comprehension), you signal that you're going to break out of the loop and jump to the exit of the function if the branch executes, meaning that the comprehension may not finish evaluating, rendering it invalid to use as a value/expression.

To see this a bit more easily, take this tweak for example:

f = ->
  numbers = for x in [1..5]
    return null if bad x
    x

In this case, CoffeeScript will error with "cannot use a pure statement in an expression". You can't use numbers as an array, because it may complete the iteration over the range to produce a value, and it may not, and we don't know in advance which will be true.

In other words, the statement-oriented way to write this would be like:

f = ->
  result = []
  for x in [1..5]
    return null if bad x
    result.push x
  result

And the expression-oriented way to write this would be more like:

f = ->
  valid = true
  numbers = for x in [1..5]
    valid = false if bad x
    x
  if valid then numbers else null

To sum up — every expression in CoffeeScript needs to be able to run to completion to produce a complete value. If you can break out of the middle of it, then it's a statement, and the value can't be used.

[edemaine]

@jashkenas Thanks for the detailed explanation! That makes a lot of sense. And now I also see that the documentation at least mentions that this is the case (though we could try to add a bit of justification):

There are a handful of statements in JavaScript that can’t be meaningfully converted into expressions, namely break, continue, and return. If you make use of them within a block of code, CoffeeScript won’t try to perform the conversion.

I do feel like return is a bit special, and there is a clear meaning to using it within an expression. Your numbers example won't break if return null executes, because return leaves the namespace that defines numbers, so it's not important that it wasn't assigned. Ah, but it gets messier if numbers is defined in an outer scope that's not getting left by return. I still think it would be intuitive to not assign to numbers in the case of a premature return, but maybe it's more trouble than it's worth; I think in the end this would be recreating the semantics of a throw.

[jashkenas]

Ah, but it gets messier if numbers is defined in an outer scope that's not getting left by return. I still think it would be intuitive to not assign to numbers in the case of a premature return, but maybe it's more trouble than it's worth; I think in the end this would be recreating the semantics of a throw.

Exactly. Like this:

numbers = []

populateNumbers = -> 
  numbers = for x in [1..10]
    return null if bad x
    x

populateNumbers()

What is the value of numbers now? Maybe null, maybe [], maybe undefined, maybe a half-filled array? Better to avoid the issue by not mixing expressions and statements.

If we were doing it again from scratch (and perhaps not based on JS) — I would love to see a completely expression-oriented language that is able to have both non-local control flow statements (return, jump to label, throw) while keeping the larger expressions that they emanate from intact as values. But I'm not sure how to do it.

[edemaine]

FWIW, the semantics that I think makes sense is that, if the right-hand side fails to evaluate, then the assignment doesn't happen at all. So in the bad case, numbers should remain the previous value []. This is already how exceptions work (throw can be a CS expression). Indeed, I could simulate the behavior I intend in current CS like so:

class ReturnException extends Error
  constructor: (value) ->
    super()
    @name = 'ReturnException'
    @value = value
numbers = []
populateNumbers = ->
  try
    numbers = for x in [1..10]
      throw new ReturnException null if bad x
      x
  catch e
    if e instanceof ReturnException
      e.value
    else
      throw e

Back to adding this to CS, I think the main argument against adding support for this idea is that it'd be really hard when the loop comprehension compiles to a closure. For example:

populateNumbers = ->
  numbers.push ...(for x in [1..10]
    #return null if bad x
    x
  )

compiles to

populateNumbers = function() {
  var x;
  return numbers.push(...((function() {
    var i, results;
    results = [];
    for (x = i = 1; i <= 10; x = ++i) {
      //return null if bad x
      results.push(x);
    }
    return results;
  })()));
};

and now it's not so easy to just add a return inside the for loop.