`.yardopts` refuses filenames containing non-ASCII characters

[noraj]

.yardopts refuses filenames containing non-ASCII characters

Steps to reproduce

Taking the following file:

module Test
  VERSION = '0.0.1'
end

Name it testé.rb and run yard doc testé.rb --no-yardopts: everything is fine.

Now create a file named .yardopts with the following content

testé.rb

It gives the following error (see Actual Output).

Actual Output

/home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/logging.rb:104:in `block in progress': incompatible character encodings: UTF-8 and ASCII-8BIT (Encoding::CompatibilityError)
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/logging.rb:103:in `synchronize'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/logging.rb:103:in `progress'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/logging.rb:81:in `capture'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/parser/source_parser.rb:45:in `parse'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/parser/source_parser.rb:371:in `parse_in_order'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/parser/source_parser.rb:114:in `block in parse'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/logging.rb:182:in `enter_level'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/parser/source_parser.rb:113:in `parse'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard.rb:20:in `parse'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/cli/yardoc.rb:259:in `block in run'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/serializers/yardoc_serializer.rb:56:in `lock_for_writing'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/registry_store.rb:202:in `lock_for_writing'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/registry.rb:210:in `lock_for_writing'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/cli/yardoc.rb:258:in `run'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/cli/command.rb:14:in `run'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/cli/command_parser.rb:72:in `run'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/cli/command_parser.rb:54:in `run'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/bin/yard:13:in `<top (required)>'
        from /home/noraj/.asdf/installs/ruby/3.2.2/bin/yard:25:in `load'
        from /home/noraj/.asdf/installs/ruby/3.2.2/bin/yard:25:in `<main>'

Expected Output

No error.

Environment details:

• OS: ArchLinux
• Ruby version: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux]
• YARD version: yard 0.9.34

(Un)related issues

All issues I found for that error are from Ruby 1.9.3 from 2013. But since Ruby 2.0 there is no need to provide an encoding comment to specify the source encoding. Generally there is no issue with encoding with yard even if my files are full of non-ASCII characters in the comments, content and file names. There only issue is when the file name contains non-ASCII chars in .yardopts.

Debug

This where yard seems to crash

https://github.com/lsegal/yard/blob/2d197a381c5d4cc5c55b2c60fff992b31c986361/lib/yard/logging.rb#L104

However, running print("\e[2K\e[?25l\e[1mtesté\e[0m\r") does not seem to crash.

[noraj]

Weird, I pushed this .yardopts file https://github.com/IDLFAC/CyberMots-ILC/blob/a16e3a4f4df91c3a1d4d4e883f873cc616a10034/.yardopts that makes yard crash locally with ruby 3.2.2 and yard 0.9.34 but I gave my github repository to rubydoc.info https://www.rubydoc.info/github/IDLFAC/CyberMots-ILC/ and it successfully built it with the same version Generated on Sun Nov 5 19:25:34 2023 by [yard](http://yardoc.org/) 0.9.34 (ruby-3.2.2)..

Is yard info using a special / patched version of yard? A different configuration? Using last commit from master and not the release? It must take my .yardopts file into consideration because for the gem when there was no .yardopts the documentation was blank https://www.rubydoc.info/gems/cybermots because the library folder is non standard.

[lsegal]

I can't seem to reproduce the issue in ruby 3.3.0 on my machine. Given that rubydoc.info also does not have the issue you're reporting, could there be something in your environment? I'm thinking about marking this as closed given the failure to repro.

[noraj]

I can reproduce on ruby 3.3.0, both the minimal example and running bundle exec yard doc on my project.

[noraj]

I made a minimal repository to reproduce: https://github.com/noraj/yard-issue-1517

[lsegal]

Thanks for the repro. This is now fixed and will be in the next release.