Scraping does not seem to use Unicode/UTF-8, Japanese gets garbled

[vermeeren]

Describe the bug

Bookmark scraping appears to not use unicode, resulting in garbled characters with for example Japanese.

The PHP default_charset is set to UTF-8, Debian defaults. Japanese works fine with file syncing and in other parts of Nextcloud.

To Reproduce

Add bookmark https://www.youtube.com/watch?v=OoM0ikOi1v4 with web scraping turned on.

【Blender】初心者向け！Blender超入門講座　～簡単なセルルックのうさぎのキャラクターを作ろう！～

Seems like the values are inserted into the database garbled.

# from psql command line
nextcloud=# select title from oc_bookmarks;

ã\u0080\u0090Blenderã\u0080\u0091å\u0088\u009Då¿\u0083è\u0080\u0085å\u0090\u0091ã\u0081\u0091ï¼\u0081Blenderè¶\u0085å\u0085¥é\u0096\u0080è¬\u009B座ã\u0080\u0080ï½\u009Eç°¡å\u008D\u0098ã\u0081ªã\u0082»ã\u0083«ã\u0083«ã\u0083\u0083ã\u0082¯ã\u0081®ã\u0081\u0086ã\u0081\u0095ã\u0081\u008Eã\u0081®ã\u0082­ã\u0083£ã\u0083©ã\u0082¯ã\u0082¿ã\u0083¼ã\u0082\u0092ä½\u009Cã\u0082\u008Dã\u0081\u0086ï¼\u0081ï½\u009E

PostgreSQL database using UTF8 for encoding and en_US.UTF-8 for collate and ctype.

Expected behavior

【Blender】初心者向け！Blender超入門講座　～簡単なセルルックのうさぎのキャラクターを作ろう！～ 

Screenshots

Render from the bookmarks UI in firefox.

Desktop (please complete the following information):

• OS: Debian Linux
• Browser: Firefox
• Version: ESR 128

Server (please complete the following information):

• OS: Debian bookworm
• HTTP server: nginx
• Database: PostgreSQL 15
• PHP version: 8.2 FPM
• Nextcloud version: 30.0.13
• Bookmarks app version: 15.1.3
• Activated Nextcloud Apps: See https://github.com/nextcloud/server/issues/54134#issuecomment-3164517293
• Nextcloud configuration: See https://github.com/nextcloud/server/issues/54134#issuecomment-3164517293
• Nextcloud external user backend: none

Additional context

Web server error log

Nothing shows up in logs.

Nextcloud log (nextcloud/data/nextcloud.log)

Nothing shows up in logs.

Browser log

Not sure about this.

[github-actions[bot]]

Hello :wave:

Thank you for taking the time to open this issue with the bookmarks app. I know it's frustrating when software causes problems. You have made the right choice to come here and open an issue to make sure your problem gets looked at and if possible solved. I'm Marcel and have been maintaining this software the last few years. I currently work for Nextcloud but maintain this app in my free time, because it is not an official Nextcloud product. My day job at Nextcloud is pretty awesome but sadly leaves me with less time for side projects like this one than I used to have. I still try to answer all issues and if possible fix all bugs here, but it sometimes takes a while until I get to it. Until then, please be patient. Note also that GitHub is a place where people meet to make software better together. Nobody here is under any obligation to help you, solve your problems or deliver on any expectations or demands you may have, but if enough people come together we can collaborate to make this software better. For everyone. Thus, if you can, you could also look at other issues to see whether you can help other people with your knowledge and experience. If you have coding experience it would also be awesome if you could step up to dive into the code and try to fix the odd bug yourself. Everyone will be thankful for extra helping hands! One last word: If you feel, at any point, like you need to vent, this is not the place for it; you can go to the forum, to twitter or somewhere else. But this is a technical issue tracker, so please make sure to focus on the tech and keep your opinions to yourself. (Also see our Code of Conduct. Really.)

I look forward to working with you on this issue Cheers :blue_heart:

[marcelklehr]

Hi @vermeeren Thank you for taking the time to give feedback! I can confirm the issue and will work on a fix.