jslex

jslex copied to clipboard

jslex

Lexical analyzer JavaScript library. Similar to lex and flex.

Developed by

• Jim R. Wilson (jimbojw)

GitHub repository

• http://github.com/jimbojw/jslex/

License

This project is released under The MIT License.

Usage

The best way to understand how to use jslex is via illustrative example.

First, create a lexer:

var lexer = new jslex( {
    "start": {
        "[0-9]+": function() {
            return parseInt(this.text, 10);
        },
        "[-+\n]": function() {
            return this.text;
        },
        "[ \t\r]": null,
        ".": function() {
            throw "Invalid character '" + this.text + "' (line:" + (1 + this.line) + ", column:" + this.column + ")";
        }
    }
} );

The above lexer has one state, called "start", which contains 4 patterns:

• one for numbers,
• one for plus, minus, or new lines,
• one for uninteresting whitespace characters, and
• one for anything else.

To use the lexer on an input string, you can use the lex() method:

function callback( token ) {
  // Do something with returned token
}

lexer.lex( "1 + 2", callback );

In the above example, the callback() function will be called three times:

• first with the number 1,
• then with the string "+", and
• lastly with the number 2.

A very common use case for the lexer is to simply collect all the tokens in an input stream. For this, use the collect() function:

var tokens = lexer.collect( "1 + 2" );

After executing the above snippet, the tokens variable would have three elements: [1, "+", 2].