Add Ripper-based parser and ast printer script for comparison against tree-sitter-ruby output.

[aibaars]

This pull request adds a script named script/print-ast.rb which uses the Ruby::Ripper parser to parse a Ruby file and prints its AST in the same form as tree-sitter. It can be used to generate expected output for inclusion in test cases.

The PR also adds the script script/compare-ast.sh which can be used to compare the AST's produced by script/print-ast.rb against the ones produced by tree-sitter parse. These "back-to-back" comparison of tree-sitter-ruby against Ruby::Ripper can help find cases where the two parsers diverge, likely indicating bugs in the tree-sitter-ruby grammar.

The PR also integrates the "back-to-back" tests to the CI job to test for regressions. This makes the CI job quite a bit slower . It now takes 7 minutes to complete instead of 1 minute. However, I think the improved regression testing is worth it.

@maxbrunsfeld @nickrolfe

Checklist:

• [x] All tests pass in CI.
• [x] There are sufficient tests for the new fix/feature.
• [x] Grammar rules have not been renamed unless absolutely necessary.
• [x] The conflicts section hasn't grown too much.
• [x] The parser size hasn't grown too much (check the value of STATE_COUNT in src/parser.c).

[mbj]

BTW: I consider the https://github.com/whitequark/parser AST as the supreme ruby tooling AST compared to the ripper one, especially as its more stable and better documented.

It may make sense to investigate it as an addition/replacement for ripper based tests in the future.

[aibaars]

BTW: I consider the https://github.com/whitequark/parser AST as the supreme ruby tooling AST compared to the ripper one, especially as its more stable and better documented.

It may make sense to investigate it as an addition/replacement for ripper based tests in the future.

That's a good point. The reason I picked Ripper is because it is just an external interface to the actual MRI parser. The API and documentation of Ripper are indeed rather terse. I would definitely prefer using whitequark/parser for anything more complex than printing an AST ;-)