tomoya ishida

This approach changes the way to write metaprogramming comment. Example in https://ruby.github.io/rdoc/RDoc/Parser/Ruby.html#class-RDoc::Parser::Ruby-label-Metaprogrammed+Methods indicates that newline between metaprogramming comment and metaprogramming code is allowed, and this pull request changes it. ```ruby...

> Why don't we make Prism a dependency and just remove `RipperStateLex`? RipperStateLex is still used in `parser/ruby.rb` for parsing, so we can't remove it now. This new tokenizer doesn't...

> And this PR will add another tokenizer using Prism. Is this correct? Yes. And two of the Ripper based parser/tokenizer will be unmaintaind/keep unchanged until we drop it. >...

I think syntax highlight is not a bottleneck. Measurement in my environment was: | scenario | time | | --- | --- | | master(prism_ruby) | 31.6 sec | |...

`BigDecimal#add` is in the same situation ```ruby BigDecimal('0.3333333333333333333333333333333333').sub(1/3r, 40) #=> -0.333333e-34 # 6 digits, not 40 digits ```

This code depends on existence of `globals.rdoc` and also on its document structure. We need a way to detect file path/structure change in `ruby/ruby` Can you add a test that...

There is a related pull request https://github.com/ruby/rdoc/pull/1363 that restrict the highlight only to method signature part. It solve method entries not-aligned problem in another way. I think it's both fine...

> Argument +single_encoding+, if true, encoding will be determined from the first row. I think it should scan every row and header. The below is an example CSV that encoding...

When encoding is known, `to_csv(encoding: Encoding::UTF_8)`, your fast path improvement is significant :+1: For `single_encoding: true` option, it needs discussion. If there is a way to moderately improve without that...