feat: ability to test rule errors and invalid schemas

[bmish]

Prerequisites checklist

• [x] I have read the contributing guidelines.

What is the purpose of this pull request? (put an "X" next to an item)

[ ] Documentation update [ ] Bug fix (template) [ ] New rule (template) [ ] Changes an existing rule (template) [ ] Add autofix to a rule [ ] Add a CLI option [X] Add something to the core [ ] Other, please explain:

What changes did you make? (Give an overview)

Implements this RFC:

• https://github.com/eslint/rfcs/pull/103

Fixes #13434.

TODO:

• [x] Wait for RFC approval
• [x] Copy changes to flat-rule-tester.js
• [x] Add rule tester tests
• [x] More documentation
• [ ] Add tests for fatal error when empty schema that's invalid, present schema that has one validation error, and present schema that has multiple validation errors (https://github.com/eslint/eslint/pull/17356)

Is there anything you'd like reviewers to focus on?

[netlify[bot]]

✅ Deploy Preview for docs-eslint canceled.

Name	Link
🔨 Latest commit	2b39ad336b6bd34ba6b1dab4b702f0ed68b1fa5c
🔍 Latest deploy log	https://app.netlify.com/sites/docs-eslint/deploys/64bf1aa78cd5c20008cb3540

[github-actions[bot]]

Hi everyone, it looks like we lost track of this pull request. Please review and see what the next steps are. This pull request will auto-close in 7 days without an update.

[Rec0iL99]

WIP. The implementation proposal was approved recently. Don't close

[bmish]

This is ready for review.

[fasttime]

Marking as accepted, as this implements a merged RFC.

[nzakas]

@bmish can you circle back around to finish this up based on @mdjermanovic's feedback?

[nzakas]

@mdjermanovic this one is ready for you to look over again

[fasttime]

In #17475 we have changed the logic in RuleTester and FlatRuleTester to avoid creating empty suites for valid or invalid tests. In other words, describe will be only called if there are any its to call inside of it.

I was thinking that the same change should be applied to fatal tests: describe("fatal", ... should be only called if any fatal tests were specified.

[nzakas]

@bmish what's the status on this? Looks like it hasn't been updated in a while. Are you looking to get this into v9?

[bmish]

@nzakas this isn't a breaking change so I don't think it needs to be included in the initial version of ESLint v9. Will aim to get it into a later version.

[nzakas]

Sounds good. I'll take it off my radar for now.

[nzakas]

@bmish what are your thoughts on this? There are a lot of conflicts now. Should we just close this?

[bmish]

I'll close this for now since there are a number of issues to resolve still. Anyone else is welcome to take it over.