Discuss
  • Generator Arrow Functions

Generator Arrow Functions

# Brandon Benvie (10 years ago)

Currently, it's not allowed that arrow functions be generators. I did a bit of searching and couldn't find the original reasoning behind this. *() => {} doesn't seem to be a problematic grammar since foo * () => {} isn't valid. The problem I do see is the mismatch between the generator class hierarchy and the fact that arrow functions don't have prototypes. I think this could be worked around somehow though.

The use case I've started running into a lot is using Task.js with methods:

class Foo {
   foo() { //--> Promise
     return Task.spawn(*() => {
       const value = yield this.get(this.base + "/foo");
       if (yield this.bar(value)) {
         return true;
       }
     });
   }

   bar(value) { //--> Promise
     /***/
   }

   get(url) { //--> Promise
     /***/
   }
}

Without generator arrows, I'm back to using var self = this or .bind(this).

# Allen Wirfs-Brock (10 years ago)

Everybody should probably review esdiscuss.org/topic/why-do-generator-expressions-return-generators where we discussed this before.

It wasn't that there was necessarily anything bad about them, but there also didn't seem to be a strong enough case made in that thread to justify the work necessary to add them at this late data.

As you mention, there are issues with arrow functions not being constructors, although it probably could be dealt with similarly to how generator comprehensions are handled (generator comprehensions are essentially treated in the spec. as a special form of generator function).

I also need to think a bit about whether there might be any this binding issues.

# Axel Rauschmayer (10 years ago)

This is relevant, too: esdiscuss.org/topic/function-declarations-with-lexical-this

I’d still argue that generator arrow functions make more sense than generator function declarations.

Axel

# David Bruant (10 years ago)

I don't have a strong opinion in this debate, but I've seen something relevant in Angus Croll's slides recently:

let idGenerator = (id=0) => () => id++;

let nextFrom1000 = idGenerator(1000);
nextFrom1000(); // 1000
nextFrom1000(); // 1001
# Claus Reinke (10 years ago)

Everybody should probably review esdiscuss.org/topic/why-do-generator-expressions-return-generators where we discussed this before.

which suggests using generator expressions as arrow bodies to make generator functions with arrows

() => (for (y of gen) y)

What I don't understand is why generator expressions are not used as the only way to create generators, leaving 'function' alone. There would be

  • comprehension-style generator expressions, with implicit yield (for (...) ...)

  • block-style generator expressions, with explicit yield (do*{ ... yield value ... })

and generator functions would be build from generator expressions and functions (arrows or classic). No need to mix up functions and generators. At least none I can see...

Claus

# Brendan Eich (10 years ago)

Claus Reinke wrote:

What I don't understand is why generator expressions are not used as the only way to create generators, leaving 'function' alone.

We have been over this before: to support flows that for-of loops cannot expression, specifically coroutine libraries such as taskjs.org.

# Ѓорѓи Ќосев (10 years ago)

On Thu 14 Nov 2013 11:16:22 PM CET, Brendan Eich wrote:

Claus Reinke wrote:

What I don't understand is why generator expressions are not used as the only way to create generators, leaving 'function' alone.

We have been over this before: to support flows that for-of loops cannot expression, specifically coroutine libraries such as taskjs.org.

The suggested topics seem to be ignoring that use case (using the result of the yield expression) and only considering the iterator case. The suggestion:

() => (for (y of gen) y)

isn't going to work for generators that need the values of the yield expression (like in task.js), right?

spawn(() *=> yield (yield this.user.getPendingFriendship(friendId)).setAccepted(true));

From what I've seen, async generators are so popular (at least in node-land) that people are building entire frameworks1 and library ecosystems2 based on them even though they're only available behind a flag, in an unstable version of node. Even more, a dedicated transpiler was written3 to support transpiling just generators, with the async use case in mind.

Right now from where I stand, its almost as if the other use case of generators (as iterators) is completely unimportant.

So I really don't see how there isn't a strong enough case for generator arrow functions.

# Kevin Smith (10 years ago)

From the examples I've seen so far in this discussion, it's likely that

what is wanted isn't generator arrows, as much as syntactic support for asyc functions. Such a function, would, among other things, insure that all exceptions are converted to rejections, even exceptions occurring "in the head" (as can occur with argument destructuring and default parameter evaluation).

# Domenic Denicola (10 years ago)

On 15 Nov 2013, at 08:47, "Kevin Smith" <zenparsing at gmail.com> wrote:

From the examples I've seen so far in this discussion, it's likely that what is wanted isn't generator arrows, as much as syntactic support for asyc functions. Such a function, would, among other things, insure that all exceptions are converted to rejections, even exceptions occurring "in the head" (as can occur with argument destructuring and default parameter evaluation).

+1. Although, the outer-this-capturing nature of arrow functions would probably be nice for some async functions too.

# Brandon Benvie (10 years ago)

It would be great to have await, but in the meantime having generator functions would help male async methods tolerable. Await is ES7 at the earliest, generator arrow functions could be in ES6.

# Axel Rauschmayer (10 years ago)

It would be great to have await, but in the meantime having generator functions would help male async methods tolerable. Await is ES7 at the earliest, generator arrow functions could be in ES6.

Couldn’t arrow generator functions replace generator function declarations? In other words: is the dynamic this in generator function declarations ever useful?

Then we’d have a nice symmetry in ES6:

  • non-method function = const + arrow function.
  • method = concise method definition
  • non-method generator function = const + arrow generator function.
  • generator method = concise generator method definition

That would make the async programming code more compact, too (I’m assuming a nullary paren-free arrow variant and I prefer the asterisk after the arrow):

spawn(=>* {
    var data = yield $.ajax(url);
    $('#result').html(data);
    var status = $('#status').html('Download complete.');
    yield status.fadeIn().promise();
    yield sleep(2000);
    status.fadeOut();
});

Versus:

spawn(function*() {
    var data = yield $.ajax(url);
    $('#result').html(data);
    var status = $('#status').html('Download complete.');
    yield status.fadeIn().promise();
    yield sleep(2000);
    status.fadeOut();
});

[Example taken from task.js website.]

# Rick Waldron (10 years ago)

On Fri, Nov 15, 2013 at 11:34 AM, Axel Rauschmayer <axel at rauschma.de> wrote:

It would be great to have await, but in the meantime having generator functions would help male async methods tolerable. Await is ES7 at the earliest, generator arrow functions could be in ES6.

Couldn’t arrow generator functions replace generator function declarations? In other words: is the dynamic this in generator function declarations ever useful?

As useful as it is in non-generator function declarations and expressions. I agree that a generator arrow function adds balance, but replacement of generator function declarations contradicts a balance.

Then we’d have a nice symmetry in ES6:

  • non-method function = const + arrow function.
  • method = concise method definition
  • non-method generator function = const + arrow generator function.
  • generator method = concise generator method definition

Let me counter with:

  • function declaration, generator function declaration
  • function expression, generator function expression
  • concise method, concise generator method
  • arrow function (, generator arrow function)

That would make the async programming code more compact, too (I’m assuming a nullary paren-free arrow variant and I prefer the asterisk after the arrow):

To be clear, this preference is inconsistent with all other generator forms where the asterisk is before the params, per Brandon's original examples.

# Kevin Smith (10 years ago)

It would be great to have await, but in the meantime having generator functions would help male async methods tolerable. Await is ES7 at the earliest, generator arrow functions could be in ES6.

Adding new features to ES6 at this point in time? I don't know...

Besides, with async functions, the use case established in this thread for generator arrows basically disappears. It's probably better not to introduce convenience features that will be made obsolete by the next version of the language.

# Axel Rauschmayer (10 years ago)

Let me counter with:

function declaration, generator function declaration function expression, generator function expression concise method, concise generator method arrow function (, generator arrow function)

I don’t mind generator function declarations, but I personally will not use function declarations under ES6 (or at least try out const arrows and see how they feel). And I’ll do the same with generator function declarations, should we get generator arrow functions. If it works out then function declarations are a legacy feature (for me).

That would make the async programming code more compact, too (I’m assuming a nullary paren-free arrow variant and I prefer the asterisk after the arrow):

To be clear, this preference is inconsistent with all other generator forms where the asterisk is before the params, per Brandon's original examples.

Ah, missed that. Thought it was *=>. I don’t mind either way, my (admittedly weak) mnemonic would be “asterisk after function-defining token”.

# Claude Pache (10 years ago)

Le 15 nov. 2013 à 17:59, Rick Waldron <waldron.rick at gmail.com> a écrit :

On Fri, Nov 15, 2013 at 11:34 AM, Axel Rauschmayer <axel at rauschma.de> wrote:

(...)

That would make the async programming code more compact, too (I’m assuming a nullary paren-free arrow variant and I prefer the asterisk after the arrow):

To be clear, this preference is inconsistent with all other generator forms where the asterisk is before the params, per Brandon's original examples.

The other point of view is that this preference is consistent with other generator forms where the asterisk is after the token that defines the general role of the construct as a procedure (either the function keyword, or the => token). Personally, I tend to read function*as a unit meaning "generator function", and so would I for =>*.

# Ѓорѓи Ќосев (10 years ago)

On 11/15/2013 06:07 PM, Kevin Smith wrote:

Besides, with async functions, the use case established in this thread for generator arrows basically disappears. It's probably better not to introduce convenience features that will be made obsolete by the next version of the language.

Can someone explain how would that use case disappear with "async functions"?

If by "async functions" you mean something that would allow await 1 - as far as I understand, this already works with generators:

function* getAllUserDataLazily() {
    for (const userName of users) {
        yield getUserData(userName);
    }
}

function* outerGenerator() {
    for (const dataPromise of getAllUserDataLazily()) {
        console.log(yield dataPromise);
    }
}

The inner getAllUserDataLazily generator yields promise values to the outerGenerator, then pauses. The outer generator is run by the async task runner and it yields the promise, then it also pauses. The task runner's code now waits for the promise to resolve. Once the promise resolves, it resumes the outerGenerator. Once the outerGenerator resumes, it goes back to the beginning of the loop and resumes getAllUserDataLazily. Basically, as long as the outermost generator is run by an async runner, both lazy and async execution of all nested generators/iterators is possible.

Another benefit is that this works with primitives other than promises. Current libraries will love that, as not all are based on promises. By the time ES7 is released, a large ecosystem of async generators will already be in place. Infact, its already here, right now - and ES6 isn't even finalized yet. This is a guess, but I'd say that at that point, a new async functions feature will be seen as unnecessary and async generators as "good enough". IMO, the cat is already out of the bag.

# Brendan Eich (10 years ago)

I've added generator arrows to the TC39 meeting agenda for next week.

# Kevin Smith (10 years ago)

Basically, as long as the outermost generator is run by an async runner, both lazy and async execution of all nested generators/iterators is possible.

Sure - it's the "async runner" part though (e.g. spawn), which is the use case presented here.

async function F(p1, p2, ...pN) { /*...*/ }

Would (very) approximately de-sugar to something like:

function F(...args) {
    return spawn(this, args, function* (p1, p1, ...pN) { /* ... */ });
}
# Claus Reinke (10 years ago)

What I don't understand is why generator expressions are not used as the only way to create generators, leaving 'function' alone.

We have been over this before: to support flows that for-of loops cannot expression, specifically coroutine libraries such as taskjs.org.

Which is why I keep suggesting block-style generator expressions in addition to comprehension-style generator expressions. The equivalent of today's

function*() { ... yield value ... }

would be

function() { return do* { ... yield value ... }}

or, if 'function' peculiarities don't matter, the simpler

() => do* { ... yield value ... }

As far as I can tell, no functionality would go missing. 'function' and arrow would remain on par and function and generators would remain separate (but composable) building blocks, leading to a more modular language spec. You could keep 'function*' as syntactic sugar.

Claus

# Axel Rauschmayer (10 years ago)

Caveat: with yield*, you want generators to be more like functions than like blocks.

# Kevin Smith (10 years ago)

Heck, why not just add async functions to the agenda?

There's:

  • Promises, yay
  • A well-establish use-case, which is awkward to implement without (as the original post demonstrates)
  • Strong syntactic precedent with C#
  • Strong semantic cowpath with TaskJS
  • Strong developer interest
  • A year to work out any kinks : )

By "async function", I mean something like:

async function F(p1, p2, ...pN) {
   await G();
}

which would de-sugar to something like:

function F(...args) {
    return Task.spawn(function(p1, p2, ...pN) {
      (yield G());
    }.bind(this, ...args));
}

With a [NoNewLine] after the "async" contextual keyword, obviously.

Low-risk, high-reward?

# Brendan Eich (10 years ago)

Kevin Smith <mailto:zenparsing at gmail.com> November 16, 2013 6:56 AM

Heck, why not just add async functions to the agenda?

They are on the further-out agenda for ES7, but Object.observe is ahead, and it is driving the event-loop task/microtask specification, which async functions need too. None of this fits in ES6.

In contrast, generator arrows are relatively straightforward and might fit in ES6.

No schedule chicken, in either case, but no way do async functions + event loop full spec fit in ES6.

There's:

  • Promises, yay
  • A well-establish use-case, which is awkward to implement without (as the original post demonstrates)

More awkward without generator arrows, but survivable in any event.

  • Strong syntactic precedent with C#

Syntax is the last of it. BTW Mark was advocating function!, not async function, so there will be bikeshedding.

  • Strong semantic cowpath with TaskJS

No. That's not a scheduler + event loop spec, and the scheduler there is not the only one or even the default one people will agree to.

  • Strong developer interest

That applies to a great many things, and libraries are "go".

  • A year to work out any kinks : )

No, you're wrong -- ES6 is all but done by March, in the "fussing over typography", heading toward Ecma CC approval in June and GA approval in December. There is no "year". Any kinks would be set in stone and unchangable in a few months.

By "async function", I mean something like:

See above. You've completely ignored interoperable scheduler specification, which depends on event loop spec in full.

# Brendan Eich (10 years ago)

Claus Reinke wrote:

What I don't understand is why generator expressions are not used as the only way to create generators, leaving 'function' alone.

We have been over this before: to support flows that for-of loops cannot expression, specifically coroutine libraries such as taskjs.org.

Which is why I keep suggesting block-style generator expressions in addition to comprehension-style generator expressions.

Which is why I keep responding that blocks are not functions, we do not have expression continuations, or even block continuations, that can be captured. (Remember, block lambdas fell in spring 2012). Only functions.

This is really tedious to rehash every few months!

# Kevin Smith (10 years ago)
  • A year to work out any kinks : )

No, you're wrong --

I welcome correction as always, but I would appreciate being able to post to es-discuss without having to worry about this kind of backlash. Thanks.

# Brendan Eich (10 years ago)

Kevin Smith wrote:

I welcome correction as always, but I would appreciate being able to post to es-discuss without having to worry about this kind of backlash. Thanks.

Sorry. How about "you're mistaken"? :-|

Allen has laid out the schedule several times on this list.

Excuse my grumpy tone. List search still isn't what it ought to be. On the other hand, with no spec for the event loop and the (pluggable, what's the default) scheduler, we are pretty far from async functions being something to push for in ES6.

Do you see what I mean?

I see your point(s) about generator arrows (both less important given async functions, and late for ES6).

# Brendan Eich (10 years ago)

Brendan Eich wrote:

Sorry. How about "you're mistaken"? :-|

Wow, still grumpy when I wrote that. Yikes.

Ok, trying again: please accept my apology for laying about with the personal "Y" pronoun, the to-be verb, and "wrong". That's never productive. The "why" leading to any such conclusion is the key to useful discussion. I'll keep focused on that. Sorry, for real.

# Kevin Smith (10 years ago)

Thanks - and thanks for pointing out the dependency on a scheduling specification. <3, for real.

# Ѓорѓи Ќосев (10 years ago)

On 11/15/2013 06:18 PM, Claude Pache wrote:

The other point of view is that this preference is consistent with other generator forms where the asterisk is after the token that defines the general role of the construct as a procedure (either the function keyword, or the => token). Personally, I tend to read function* as a unit meaning "generator function", and so would I for =>*.

There is one reason (that is beyond bikeshedding) why I think the star should be together with the arrow. (I have no opinion whether it should be before or after the arrow though)

Its harder to scan whether this is a generator arrow function or a normal arrow function because the star is too far away:

someFunction(*(someArgument, anotherArgument) => {
    ... code ... 
});

compared to this form, where its immediately obvious that this is not a regular function, just by looking at the composed symbol (arrow-star)

someFunction((someArgument, anotherArgument) =>* {
    ... code ... 
});

The arbitrary length of arguments, as well as the ability to split them to multiple lines makes it potentially even harder to reliably locate the star:

someFunction(*(argument,
                          anotherArgument,
                          somethingThird) => {
                              ... code ...
});

vs

someFunction((argument,
                         anotherArgument,
                         somethingThird) =>* {
                              ... code ...
});

Here is the single-argument example, for completeness:

someFunction(*singleArgument => yield asyncOp(singleArgument));

versus

someFunction(singleArgument =>* yield asyncOp(singleArgument));

in which it looks like the star is somehow related to the argument.

If you're not convinced, you can try to find all the arrow-stars in the above code, then try to find all the arrows and determine whether they're generator arrows or regular arrows :)

# Brendan Eich (10 years ago)

Ѓорѓи Ќосев wrote:

Its harder to scan whether this is a generator arrow function or a normal arrow function because the star is too far away:

someFunction(*(someArgument, anotherArgument) => {
    ... code ...
});

compared to this form, where its immediately obvious that this is not a regular function, just by looking at the composed symbol (arrow-star)

someFunction((someArgument, anotherArgument) =>* {
    ... code ...
});

I buy it. This is what I'll propose next week as concrete syntax. It's a small point, but the rationale is "the star goes after the first token that identifies the special form as a function form." For generator functions, that token is 'function'. For arrows, it is '=>'.

# Irakli Gozalishvili (10 years ago)

+1 on arrowed generators! Have being actively using generators last month & have being wishing they could be like arrows, although syntax I've being thinking of was

(x, y) *> { .... }

BTW given that generators are used as a method on iterators wouldn't it make sense to consider syntax for generator members in a classes syntax ?? My hope is in ES6 we would just use class syntax for methods & arros for functions making "function" a legacy

# Axel Rauschmayer (10 years ago)

+1 on arrowed generators! Have being actively using generators last month & have being wishing they could be like arrows, although syntax I've being thinking of was

(x, y) *> { .... }

BTW given that generators are used as a method on iterators wouldn't it make sense to consider syntax for generator members in a classes syntax ??

It’s already in the spec: people.mozilla.org/~jorendorff/es6-draft.html#sec-generator-function-definitions

My hope is in ES6 we would just use class syntax for methods & arros for functions making "function" a legacy

My hope, too.

# Irakli Gozalishvili (10 years ago)

Oh I must have missed that, never mind then thanks Axel for pointing out.

# Claude Pache (10 years ago)

Le 18 nov. 2013 à 19:38, Brendan Eich <brendan at mozilla.com> a écrit :

Ѓорѓи Ќосев wrote:

Its harder to scan whether this is a generator arrow function or a normal arrow function because the star is too far away:

someFunction(*(someArgument, anotherArgument) => { ... code ... });

compared to this form, where its immediately obvious that this is not a regular function, just by looking at the composed symbol (arrow-star)

someFunction((someArgument, anotherArgument) =>* { ... code ... });

I buy it. This is what I'll propose next week as concrete syntax. It's a small point, but the rationale is "the star goes after the first token that identifies the special form as a function form." For generator functions, that token is 'function'. For arrows, it is '=>'.

/be

From the thread 1, I guess that parsing correctly the following thing would be obnoxious (at best)?

(a = yield/b/g) =>* {}

—Claude

# Waldemar Horwat (10 years ago)

On 11/26/2013 02:28 PM, Claude Pache wrote:

From the thread [1], I guess that parsing correctly the following thing would be obnoxious (at best)?

(a = yield/b/g) =>* {}

—Claude

Indeed.

And you can make even more obnoxious parses of the hypothetical combination of =>*, default parameters, and retroactive yield-scope:

(a = yield/"/) =>* (/"/g)

Are the two /"/'s regexps or is "/) =>* (/" a string token?

 Waldemar
# André Bargull (10 years ago)

On 11/26/2013 02:28 PM, Claude Pache wrote:

/ From the thread [1], I guess that parsing correctly the following thing would be obnoxious (at best)? />/ />/ (a = yield/b/g) =>* {} />/ />/ ---Claude / Indeed.

And you can make even more obnoxious parses of the hypothetical combination of =>*, default parameters, and retroactive yield-scope:

(a = yield/"/) =>* (/"/g)

Are the two /"/'s regexps or is "/) =>* (/" a string token?

  Waldemar

Are you sure? The (a = yield/b/g) part is parsed at first as a parenthesised expression and only later (after the => token)

reinterpreted as an ArrowFormalParameters grammar production.

  • André
# Waldemar Horwat (10 years ago)

On 11/26/2013 03:00 PM, André Bargull wrote:

On 11/26/2013 02:28 PM, Claude Pache wrote:

/ From the thread [1], I guess that parsing correctly the following thing would be obnoxious (at best)? />/ />/ (a = yield/b/g) =>* {} />/ />/ —Claude / Indeed.

And you can make even more obnoxious parses of the hypothetical combination of =>*, default parameters, and retroactive yield-scope:

(a = yield/"/) =>* (/"/g)

Are the two /"/'s regexps or is "/) =>* (/" a string token?

  Waldemar

Are you sure? The (a = yield/b/g) part is parsed at first as a parenthesised expression and only later (after the => token) reinterpreted as an ArrowFormalParameters grammar production.

  • André

Fine, so do this one instead:

(a = yield/"/g, b = yield/"/g) =>* {}

Does this generator have one or two parameters?

 Waldemar
# André Bargull (10 years ago)

On 11/27/2013 12:07 AM, Waldemar Horwat wrote:

On 11/26/2013 03:00 PM, André Bargull wrote:

On 11/26/2013 02:28 PM, Claude Pache wrote:

/ From the thread [1], I guess that parsing correctly the following thing would be obnoxious (at best)? />/ />/ (a = yield/b/g) =>* {} />/ />/ —Claude / Indeed.

And you can make even more obnoxious parses of the hypothetical combination of =>*, default parameters, and retroactive yield-scope:

(a = yield/"/) =>* (/"/g)

Are the two /"/'s regexps or is "/) =>* (/" a string token?

  Waldemar

Are you sure? The (a = yield/b/g) part is parsed at first as a parenthesised expression and only later (after the => token) reinterpreted as an ArrowFormalParameters grammar production.

  • André

Fine, so do this one instead:

(a = yield/"/g, b = yield/"/g) =>* {}

Does this generator have one or two parameters?

It depends on the surrounding environment.

Lexical token sequence in function environment: LP Ident[a] Assign Ident[yield] Div String Div Ident[g] RP

Lexical token sequence in generator environment: LP Ident[a] Assign Yield RegExp Comma Ident[b] Assign Yield RegExp RP

Reinterpreting the lexical token sequence per [14.2]:

It is a Syntax Error if the lexical token sequence matched by CoverParenthesisedExpressionAndArrowParameterList cannot be parsed with no tokens left over using ArrowFormalParameters as the goal symbol.

And applying the current (rev21) grammar production rule FormalParameters[Yield,GeneratorParameter] [14.4], gives either a generator with a single parameter (if in function environment) or a SyntaxError (if in generator environment). The SyntaxError is emitted because FormalParameters[Yield,GeneratorParameter] does not allow YieldExpressions [13.2.3], instead "yield" is treated as an IdentifierReference in function default parameters.

If the term "parsing a lexical token sequence" allows you to go back to the source character level, the result will be different, of course.

# Brendan Eich (10 years ago)

André Bargull wrote:

If the term "parsing a lexical token sequence" allows you to go back to the source character level, the result will be different, of course.

Just to let everyone know (meeting notes, coming soon, will make this clear), we agreed not to do generator arrows. Too much trouble of the kind identified here, also some future-hostility to function! wanting =>! (remember ! is a unary prefix operator). Other spellings all have

ASI or other future-hostility woes.

Sorry, Brandon. Worth a try, but no go.

# Axel Rauschmayer (10 years ago)

On 27 Nov 2013, at 3:29 , Brendan Eich <brendan at mozilla.com> wrote:

[...] also some future-hostility to function! wanting =>! (remember ! is a unary prefix operator). Other spellings all have ASI or other future-hostility woes.

What is function! ? Or will that be explained in the meeting notes, too?

# Erik Arvidsson (10 years ago)

It is one proposed syntax for async functions.