Provide function regex

[Kikobeats]

Could be possible require a external library that export a function that evaluate if the current term is a token?

Example:

WORD             require('lodash.words')

[GerHobbelt]

I see where such a thing would be handy, but when you want to do this at the macro level (rather than on the lexer rules) there's trouble using it like this, because macros are expanded into the lexer rule regexes, e.g.

// lexer macros:
WORDS  something

// lexer rules:
{WORDS}":"{WORDS}      return "MARKER";   // lexer token sent to parser

.             return "TEXT";

would be weird, language-wise, when WORDS require(lodash.words) would be code for WORDS as then the question is: what is {WORDS}:{WORDS} then as a rule to match?

If, instead, you'ld be looking something like this:

// lexer macros:
WORDS  something

// lexer rules:
{WORDS}":"{WORDS}      return "MARKER";   // lexer token sent to parser
/require(words)/       return "WORD"; // as recognized by require(words)

.             return "TEXT";

now that I can see working as one might expect.