decomment icon indicating copy to clipboard operation
decomment copied to clipboard

Error happens when decomment parses some regular expressions

Open DIYgod opened this issue 7 years ago • 10 comments

A minimal demo:

var decomment = require('decomment');
var code = 'if(true){}/\\d/.test(\'\')';
decomment(code);

Error log:

Error: Line 1: Unexpected token ILLEGAL
    at ErrorHandler.constructError (/Users/DIYgod/Code/test/node_modules/esprima/dist/esprima.js:5004:22)
    at ErrorHandler.createError (/Users/DIYgod/Code/test/node_modules/esprima/dist/esprima.js:5020:27)
    at ErrorHandler.throwError (/Users/DIYgod/Code/test/node_modules/esprima/dist/esprima.js:5027:21)
    at Scanner.throwUnexpectedToken (/Users/DIYgod/Code/test/node_modules/esprima/dist/esprima.js:5155:35)
    at Scanner.getComplexIdentifier (/Users/DIYgod/Code/test/node_modules/esprima/dist/esprima.js:5488:23)
    at Scanner.scanIdentifier (/Users/DIYgod/Code/test/node_modules/esprima/dist/esprima.js:5556:67)
    at Scanner.lex (/Users/DIYgod/Code/test/node_modules/esprima/dist/esprima.js:6223:26)
    at Tokenizer.getNextToken (/Users/DIYgod/Code/test/node_modules/esprima/dist/esprima.js:6666:84)
    at Object.tokenize (/Users/DIYgod/Code/test/node_modules/esprima/dist/esprima.js:154:36)
    at Object.parseRegEx (/Users/DIYgod/Code/test/node_modules/decomment/lib/utils.js:37:13)

DIYgod avatar Oct 31 '17 07:10 DIYgod

When Esprima doesn’t like your code, it must an invalid JavaScript ;)

vitaly-t avatar Oct 31 '17 07:10 vitaly-t

The compressed source code is like

if(true){}/\d/.test('');

Then gulp-strip-comments parses my code with toString function to a string like 'if(true){}/\\d/.test(\'\')'

Should I make an issue to gulp-strip-comments?

DIYgod avatar Oct 31 '17 07:10 DIYgod

I’m not sure, I will test it later though ;)

vitaly-t avatar Oct 31 '17 07:10 vitaly-t

Here's the test I've done, and it all looks fine:

const decomment = require('decomment');

const str = 'if(true){}/\d/.test(\'\');';

const s = decomment(str);

console.log(s);
//=> if(true){}/d/.test('');

Have you tried gulp-decomment?

vitaly-t avatar Oct 31 '17 11:10 vitaly-t

Sorry for my late reply.

The error happends in /\\d/, not /\d/

DIYgod avatar Nov 01 '17 02:11 DIYgod

You should log/chase it against the Esprima engine as the one that fails to process that piece of JavaScript.

There is nothing that can be done about it within this library.

vitaly-t avatar Nov 01 '17 05:11 vitaly-t

There, I've opened an issue: Fails on a compressed piece of code.

vitaly-t avatar Nov 01 '17 05:11 vitaly-t

Thanks!

DIYgod avatar Nov 01 '17 07:11 DIYgod

The author of Esprima claims he fixed it, but I've just released an update, and I can still reproduce the issue :(

vitaly-t avatar Aug 13 '18 01:08 vitaly-t

Esprima hasn't been published for 3 years, and now looks a bit abandoned.

I presently do not have time for rewriting this library to use a different parser.

vitaly-t avatar Mar 17 '21 21:03 vitaly-t