[Bug] RSS: Japanese video description encoding issues

[CatPlanet]

Describe the bug Despite RSS channels' output being in UTF8 japanese video descriptions are total gibberish

Steps to Reproduce Steps to reproduce the behavior:

1. Go to https://yewtu.be/feed/channel/UCcp9uRaBwInxl_SZqGRksDA
2. Cry

[unixfox]

Can you replicate the issue locally with a local instance of invidious?

[syeopite]

Seems like a behavior specific to yewtu.be. I can't replicate this locally

[RadoslavL]

Seems like a behavior specific to yewtu.be. I can't replicate this locally

I can also replicate it in invidious.fdn.fr and inv.tux.pizza.

[absidue]

I wonder if there is something between Invidious and the user on those major instances that is messing up the encoding, maybe a reverse proxy or something that is incorrectly configured?

[unixfox]

@syeopite did you try if you can replicate the issue in docker too?

[CatPlanet]

Chinese channels are affected too https://yewtu.be/feed/channel/UCoC47do520os_4DBMEFGg4A Taiwan https://yewtu.be/feed/channel/UC8ROUUjHzEQm-ndb69CX8Ww Korean https://yewtu.be/feed/channel/UC5BMQOsAB8hKUyHu9KI6yig Vietnam https://yewtu.be/feed/channel/UCOYdjTcOmle5oItjpEK0VpQ Arabic noodles https://yewtu.be/feed/channel/UCfiwzLy-8yKzIbsmZTzxDgw

Titles, names are formatted correctly, same with actual video pages (not RSS ones). It's only the RSS description issue.

[syeopite]

Invidious under docker is still able to render the RSS feeds correctly for me. But I am able to replicate this on public instances like onion.tube though.

iv.ggtyler.dev seems to be able to render the RSS feeds correctly

[absidue]

Does it work correctly on the test instance? afaik that is a default setup without any of the custom/non-standard stuff that most public instances do.

[RadoslavL]

Does it work correctly on the test instance? afaik that is a default setup without any of the custom/non-standard stuff that most public instances do.

/feed/channel/UCoC47do520os_4DBMEFGg4A doesn't work properly on the test instance. /feed/channel/UC8ROUUjHzEQm-ndb69CX8Ww doesn't work as well.

[absidue]

Maybe it's caused by the reverse proxy then?

[sefidel]

Can replicate this issue (NixOS 24.05), it's also happening on /api/v1/comments.

I've observed the same behaviour without reverse proxy (localhost also suffers from this problem). Perhaps it's something with the build system?

Using the docker image works fine.

[sefidel]

This is most likely caused by an outdated version of Crystal.

See https://github.com/crystal-lang/crystal/issues/13703 for more details. (Relevant Comment)

[kubo6472]

Possibly related, I found that the instance is okay, but the API is apparently the issue: lamarios/clipious#491

[sefidel]

The web interface doesn’t suffer from this problem since it uses the ‘contentHtml’ field, which doesn’t go through the XML parser.

In other words, I’m 99% certain that it’s related to this issue (which the instance admin should fix)