Bug: eslint-fuzzer Error: did not recognize object of type "PropertyDefinition"

[mdjermanovic]

Environment

Node version: 22.14.0 npm version: 10.9.2 Local ESLint version: main branch Global ESLint version: Operating System: windows

What parser are you using?

Default (Espree)

What did you do?

First, I modified the eqeqeq rule to crash: https://github.com/eslint/eslint/commit/6d45e72c517bf07ce6f9b29c6ab0533eff2b193e.

Then, I ran npm run test:fuzz.

What did you expect to happen?

A report about eqeqeq rule crashing and code samples where it crashes.

What actually happened?

$ npm run test:fuzz

> [email protected] test:fuzz
> node Makefile.js fuzz

Fuzzing rules [====--------------------------] 13%, 3.0s elapsed, eta 19.2s, errors so far: 14C:\projects\eslint\node_modules\ast-types\lib\types.js:669
            throw new Error("did not recognize object of type " +
            ^

Error: did not recognize object of type "PropertyDefinition"
    at Object.getFieldNames (C:\projects\eslint\node_modules\ast-types\lib\types.js:669:19)
    at getSortedChildNodes (C:\projects\eslint\node_modules\recast\lib\comments.js:53:23)
    at getSortedChildNodes (C:\projects\eslint\node_modules\recast\lib\comments.js:62:9)
    at getSortedChildNodes (C:\projects\eslint\node_modules\recast\lib\comments.js:62:9)
    at decorateComment (C:\projects\eslint\node_modules\recast\lib\comments.js:70:22)
    at decorateComment (C:\projects\eslint\node_modules\recast\lib\comments.js:82:13)
    at decorateComment (C:\projects\eslint\node_modules\recast\lib\comments.js:82:13)
    at decorateComment (C:\projects\eslint\node_modules\recast\lib\comments.js:82:13)
    at decorateComment (C:\projects\eslint\node_modules\recast\lib\comments.js:82:13)
    at decorateComment (C:\projects\eslint\node_modules\recast\lib\comments.js:82:13)

Node.js v22.14.0

Link to Minimal Reproducible Example

https://github.com/eslint/eslint/tree/repro-fuzzer-error

Participation

• [ ] I am willing to submit a pull request for this issue.

Additional comments

This was happening before https://github.com/eslint/eslint/commit/f5a35e3b7cee17cd31fc02c24c3e74b42ee202bc as well, so it isn't caused by that change.

[nzakas]

I'm not sure what to make of this. Can you provide some more context?

[mdjermanovic]

eslint-fuzzer runs core rules on random input to check if any would crash. When a rule crashes, it tries to minimize the input where the crash is reproducible. At the end, it prints out minimal examples where rules crash.

Now, somewhere in the process, eslint-fuzzer itself crashes, so instead of getting a list of failing rules with minimal reproducible examples, we get the output as in the original post (details about eslint-fuzzer crashing), which isn't useful.

[nzakas]

And you think this is a recent change?

[github-actions[bot]]

Oops! It looks like we lost track of this issue. What do we want to do here? This issue will auto-close in 7 days without an update.

[mdjermanovic]

And you think this is a recent change?

I'm not sure. It's quite possible that it hasn't been working properly for a long time, but we haven't noticed because rules don't crash.

[github-actions[bot]]

Oops! It looks like we lost track of this issue. What do we want to do here? This issue will auto-close in 7 days without an update.

[fasttime]

The package ast-types where the error occurs was last updated 5 years ago according to npm (https://www.npmjs.com/package/ast-types), whereas the PropertyDefinition node type which is causing the error was apparently added to ESTree in ES2022. That probably explains the crash, because newer syntax is not recognized despite being parsed correctly.

This can be fixed by catching the error in tools/code-sample-minimizer.js. Maybe it could be also worked around in recast (which is where ast-types is used), or by replacing recast with something else.

[mdjermanovic]

This can be fixed by catching the error in tools/code-sample-minimizer.js.

Good idea, I prepared ~https://github.com/eslint/eslint/issues/19961~ https://github.com/eslint/eslint/pull/20252. That should at least make fuzz testing useful until we figure out how to fix the code sample minimizer.

[github-actions[bot]]

Oops! It looks like we lost track of this issue. What do we want to do here? This issue will auto-close in 7 days without an update.

[jayaddison]

This can be fixed by catching the error in tools/code-sample-minimizer.js.

Good idea, I prepared #19961. That should at least make fuzz testing useful until we figure out how to fix the code sample minimizer.

Nitpick: I think you meant #20252? (I was reviewing a diff, and momentarily confused about what that was for, until I noticed the branch name for it)

[mdjermanovic]

This can be fixed by catching the error in tools/code-sample-minimizer.js.

Good idea, I prepared #19961. That should at least make fuzz testing useful until we figure out how to fix the code sample minimizer.

Nitpick: I think you meant #20252? (I was reviewing a diff, and momentarily confused about what that was for, until I noticed the branch name for it)

Yes, my mistake, it should have been https://github.com/eslint/eslint/pull/20252.