feat: Eslint JavaScript config (try 2)

[giseburt]

Fixes #2405 Closes #2414

Fixes maybe #3240 Fixes maybe #3126

Building on the great work of @andrestone in #2405, which appears to be abandoned (it happens), I've been using techniques I've used in other projects and inspired by Tokens in CDK to make the code work.

The goal of this PR is two-fold:

1. Have ESLint write the legacy and flat formats (as outlined here)
2. In order to do that, we provide ability to write JavaScript-formatted config files (generally) without losing abilities like escape hatches

ESLint File Format

The yaml option has been deprecated and replaced with fileFormat valued as an enum with the options: JSON, YAML, JAVASCRIPT_FLAT_ESM, JAVASCRIPT_OLD_CJS, JAVASCRIPT_FLAT_ESM, and JAVASCRIPT_FLAT_CJS.

All of these are tested and verified with new tests. The internals still use the old file format initially, and accept commands like addOverride() that are in reference to the old file format (overrides are replaced with the settings simply being a "flat" array, where each value naturally overrides the config before it).

Each of the options are converted to a flat-format equivalent. Particularly tricky parts were plugins and parsers, where they are now imported (or required) and used as javascript. The logic used in eslint for the legacy configs was emulated to determined the module names to use, and the imports are added then used. Note that imports are NOT added to the project, but that could be added later.

Also of note what that ignorePatterns is no ignores and the pattern no longer assumes **/, so we use a heuristic: if we don't find /, **, or ! (as the first character) in the patterns, we simple prepend **/.

There's more work I'd like to do here, like making auto-fixable options warn instead of error (so they don't break auto-save with format and auto-fix in VSCode, among others), and switching rules to use the new typescript-eslint module and ESLint Stylistic where many deprecated eslint rules have moved to. These will be in other PRs and issues though.

JavascriptFile class

The new interface is highlighted by this code from the tests:

  // make a dependencies object to track imports
  const dependencies = new JavascriptDependencies();

  // add a few imports
  const [jsdoc] = dependencies.addImport("jsdoc", "eslint-plugin-jsdoc");
  const [js] = dependencies.addImport("js", "@eslint/js");

  // create a files array to modify later
  const files: Array<string> = [];

  // make a data object
  const data = [
    {
      files,
      plugins: {
        jsdoc,
      },
      rules: {
        "jsdoc/require-description": "error",

        // insert a spread operator, value doesn't matter
        [`...${js}.blah`]: true,
        "jsdoc/check-values": "error",

        // insert a second spread operator, value doesn't matter
        [`...(${jsdoc}.fakeTest ? {"fakeTest": "warn"} : {})`]: true,
      },
    },
  ];

  // now make a file with that data, including any imports, but don't resolve it yet
  const unresolvedValue = new JavascriptRaw(`${dependencies}

export default ${JavascriptDataStructure.value(data)};
`);

  // modify the data
  files.push("**/*.js");

  // now resolve the code into value
  const value = unresolvedValue.resolve();

  console.log(value);

  expect(value).toEqual(*/shown below*/);

Here's the contents of value it creates, extracted so it'll get properly syntax highlighted:

import jsdoc from 'eslint-plugin-jsdoc';
import js from '@eslint/js';

export default [
  {
    files: [
      "**/*.js",
    ],
    plugins: {
      jsdoc: jsdoc,
    },
    rules: {
      "jsdoc/require-description": "error",
      ...js.blah,
      "jsdoc/check-values": "error",
      ...(jsdoc.fakeTest ? {"fakeTest": "warn"} : {}),
    },
  },
];

Escape-hatches and late binding

Also, late-binding along with addOverride and patching escape hatches still work. A brief demonstration:

const configFileName = "testFilename.mjs";

const file = new JavascriptFile(project, configFileName, {
  obj: {
    exportedValue: "value",
  },
  marker: true,
  allowComments: true,
  cjs: false,
});

At this point the output would simply be:

// ~~ Generated by projen. To modify, edit .projenrc.js and run "npx projen".
export default {
  exportedValue: "value",
};

Now lets patch that to include the fs and instead read the value from the file name the is in readFileName:

const [newValueToken] = file.dependencies.addImport("fs", "fs");
let readFileName = "default.txt";
file.addOverride(
  "exportedValue",
  JavascriptRaw.value(
    `${newValueToken}.readFileSync(${JavascriptDataStructure.value(
      () => readFileName
    )})`
  ).toString()
);

Since we used JavascriptDataStructure.value(() => readFileName) it will (1) insert the value as data instead of raw code, which in this case is a quoted string, and (2) read readFileName at the last possible moment.

So, we can still change it:

readFileName = "finalValue.txt";

Now the output will look like:

// ~~ Generated by projen. To modify, edit .projenrc.js and run "npx projen".
import fs from 'fs';
export default {
  exportedValue: fs.readFileSync("finalValue.txt"),
};

JavascriptFile features

• Take data structures and format them, with indention
  • Allows for easy embedding of data into structures, as most of the types of code we're going to be making are essentially configuration
• Uses stringification to Tokens, and replaces the tokens later, much like CDK Tokens (but much less sophisticated)
• Accept functions as values, and data structures can be modified after being provided and before resolve() is called for late-binding data
  • For example, one can include an array of values in the code that aren't known yet, but will be known before synth time
• Can be built upon to add new features - the (provided) JavascriptDependencies import tracking feature for example is added in just such a way
  • JavascriptDependencies can make CJS require or ESM import statements
  • JavascriptDependencies addImport call returns tokens that can be used in the code as strings for the imported values (js and jsDoc in the above example)
• Supports making Functions (both named and arrow)
• Supports all fundamental types including Dates, Objects, and Arrays (the latter serialized with indention, unless empty)
• Supports inserting spread operators in objects, quoting keys in objects (as needed), and raw javascript values in most places (providing spread operators in arrays, or calling functions, for example)

It is notably missing the ability to arbitrarily insert comments.

PS: JavascriptFile could be easily expanded to make TypeScript code as well, BTW.

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 94.55727% with 67 lines in your changes are missing coverage. Please review.

Project coverage is 96.29%. Comparing base (ad20d2c) to head (a7bbd20). Report is 58 commits behind head on main.

Files	Patch %	Lines
src/javascript/eslint.ts	89.50%	38 Missing :warning:
src/javascript-file.ts	96.40%	21 Missing :warning:
src/code-token-map.ts	96.44%	8 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3454      +/-   ##
==========================================
- Coverage   96.34%   96.29%   -0.05%     
==========================================
  Files         192      195       +3     
  Lines       37696    38944    +1248     
  Branches     3524     3711     +187     
==========================================
+ Hits        36320    37503    +1183     
- Misses       1376     1441      +65     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[mrgrain]

We should probably get the JavaScript Code stuff in first without the eslint part.

[cameroncf]

We should probably get the JavaScript Code stuff in first without the eslint part.

I came here looking for eslint9 support after going down a similar path. There are other needs that could be solved with a standalone javascript-file.ts component as well. For example, vite configs want to be in vite.config.js files. Merging in that limited change first would allow the js file component to be vetted against other downstream work in addition to the eslint work.