Emoji in Folder/Directory name

[charleslcso]

First time OC user.

I just did a fresh install of ownCloud 10.10, played with it.

Using the 3 default folders, I tried to add emoji to end of folder/directory name. No problem.

Then tried to remove the emoji. Cannot remove it.

In iOS app, tried to create a folder inside a shared folder (shared with 1 user). Used emoji in folder name. Failed.

Created a folder without emoji in the name. Success.

Expected behaviour

The emoji should be deleted.

Folder can be created.

Actual behaviour

When the emoji is deleted, Enter key is pressed on keyboard, the folder shows a non-stop spinner.

Server configuration

Ubuntu 22.04 LTS

Web server: Apache

Database: MariaDB 10.x

PHP version: 7.4.30

ownCloud version: (see ownCloud admin page) 10.10

Updated from an older ownCloud or fresh install: fresh

Where did you install ownCloud from: Raspberry Pi 4B

[phil-davis]

It works-for-me in the classic web UI. Please try logging in to the server from a browser and doing the same sot of thing - rename a folder to include an emoji in the name, then rename to remove the emoji from the name. And confirm if that works.

[phil-davis]

And please comment here with the emoji that you are trying to use. Maybe it is an emoji from some complex/special set, and I would need to use the same emoji to see the problem.

[charleslcso]

Hello, I've captured a short movie.

https://drive.google.com/file/d/14L2h461g6Jyl2PYCoTXkeVGCGqS86HOi/view?usp=sharing

Please let me know what else I can provide.

[phil-davis]

@charleslcso can you paste the emoji(s) that you are using into a comment in this issue please. I am having trouble reproducing your problem.

[charleslcso]

I tried 2 emojis:

3️⃣ 🚥

[phil-davis]

I wonder what happens on Raspberry Pi with MariaDb?

config/config.php should have:

  'mysql.utf8mb4' => true,

And MariaDb (and MySQL) should be a version that supports the longest 4-byte UTF-8 characters.

Can you find exactly the version of database that is running?

And what mysql.utf8mb4 is set to in config/config.php?

And what is in the logfile (somewhere like data/owncloud.log ?

Maybe this is a problem with not having full UTF-8 support.

[charleslcso]

On 24 Jul 2022, at 10:12 PM, Phil Davis @.***> wrote:

I wonder what happens on Raspberry Pi with MariaDb?

config/comfig.php should end up with:

'mysql.utf8mb4' => true,

And MariaDb (and MySQL) should be a version that supports the longest 4-byte UTF-8 characters.

Yes, it is.

'dbtype' => 'mysql', 'mysql.utf8mb4' => true,

Can you find exactly the version of database that is running?

'version' => '10.10.0.3',

And what mysql.utf8mb4 is set to in config/config.php?

true

And what is in the logfile (somewhere like data/owncloud.log ?

Ah! Just when I try to remove the emoji in directory name, owncloud.log adds an empty line.

https://drive.google.com/file/d/14LOrFmee2ic0bmmNI7HxAcLcI2jhYW-c/view

Maybe this is a problem with not having full UTF-8 support.

— Reply to this email directly, view it on GitHub https://github.com/owncloud/core/issues/40232#issuecomment-1193326522, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAD7CWNY7ZDVDXHX6MZAWOLVVVFL3ANCNFSM54PBM6GA. You are receiving this because you were mentioned.

[charleslcso]

Any code change to get more logging?

Another finding: file name with UTF-8 is not handled properly too. I placed a file with only UTF-8 character in filename into ownCloud's folder in Mac, and it won't sync with ownCloud's Desktop App. Spinner in folder icon and file icon for a long time.

ownCloud Mac Desktop App details: ownCloud 2.10.1 (build 7187) 82eefd Apr 5 2022 13:24:21 Libraries Qt 5.12.11, OpenSSL 1.1.1k 25 Mar 2021 Using virtual files plugin: suffix osx-21.4.0

I cannot delete directories with emoji in the name (mobile app nor classic web ui). Is it not ok to just remove the directives from command line, without ruining MySQL tables?

[charleslcso]

Hi @phil-davis any advice?

[phil-davis]

Hi @phil-davis any advice?

Not really - I can't reproduce your problem, and I don't have an RPi system (or similar) to try to reproduce the whole setup.

Anyone else with a good idea, please comment!

[charleslcso]

Thank you.

Is it safe for me to remove folders in command line? I hope this action won't destroy data in tables....please let me know.

[pako81]

You can physically remove the affected folders from storage and then run an occ files:scan on a path or user basis in order to update the oC filecache.

Anyway, support for 4-byte Unicode is already there in newer oC versions along with minimum MySQL version of 5.7.

It would be interesting to see the browser's network tab when performing the rename operation. Additionally, it may be worth to increase the oC log level to debug to check if additional info are being logged.

[charleslcso]

Ok. I'll try that.

I can confirm that OC doesn't work with UTF-8 characters. At least not in my case.

The files are either blacklisted, or service unavailable. The same .mp4 file, using all English characters will be OK with OC, but mixing the filename with UTF-8 characters, no go with OC.

If anyone wishes to try, you can use the following characters:

• 3️⃣🫣😜
• 你好嗎123

[charleslcso]

Turned on DEBUG logging, and captured web browser's Console Network area.

OC will write 2 blank, new lines when I upload files with UTF-8 characters in it.

---

{"reqId":"3c1a4fe9-8c74-48b3-9acf-fb3d5e26c132","level":0,"time":"2022-08-04T23:56:38+08:00","remoteAddr":"192.168.1.3","user":"root","app":"OC\\Authentication\\Token\\DefaultTokenProvider::updateToken","method":"PROPFIND","url":"\/remote.php\/dav\/files\/root\/","message":"updating token 693, last check is now 1659627809"}
{"reqId":"3c1a4fe9-8c74-48b3-9acf-fb3d5e26c132","level":0,"time":"2022-08-04T23:56:38+08:00","remoteAddr":"192.168.1.3","user":"root","app":"OC\\Authentication\\Token\\DefaultTokenProvider::getToken","method":"PROPFIND","url":"\/remote.php\/dav\/files\/root\/","message":"token bdb004e439009660030146fce4e410bcf6d2b9ecf8ea427f11b340fbd50012702046a7eb8f2e89fb61b70d32193d5a3d9936434fd77decc4726c9af8e9055b9d does not exist"}


{"reqId":"z62aEvp4uH94VQAi4Byi","level":0,"time":"2022-08-04T23:56:39+08:00","remoteAddr":"192.168.1.3","user":"root","app":"OC\\User\\Session::validateToken","method":"GET","url":"\/index.php\/apps\/files\/ajax\/getstoragestats.php?dir=%2F","message":"token bf25657469a6686af90d1361f4db1ab27b57e9e23eb1f89835ae93d937e0ea38ad59ae1ef5f7bc534d424d8c91e26374c9cf9b7f6d22845c7fa1ee270be1bd21 with token id 643 found, validating"}
{"reqId":"z62aEvp4uH94VQAi4Byi","level":0,"time":"2022-08-04T23:56:39+08:00","remoteAddr":"192.168.1.3","user":"root","app":"OC\\User\\Session::validateToken","method":"GET","url":"\/index.php\/apps\/files\/ajax\/getstoragestats.php?dir=%2F","message":"token bf25657469a6686af90d1361f4db1ab27b57e9e23eb1f89835ae93d937e0ea38ad59ae1ef5f7bc534d424d8c91e26374c9cf9b7f6d22845c7fa1ee270be1bd21 with token id 643 found, validating"}
{"reqId":"z62aEvp4uH94VQAi4Byi","level":0,"time":"2022-08-04T23:56:39+08:00","remoteAddr":"192.168.1.3","user":"root","app":"OC\\User\\Session::validateToken","method":"GET","url":"\/index.php\/apps\/files\/ajax\/getstoragestats.php?dir=%2F","message":"token bf25657469a6686af90d1361f4db1ab27b57e9e23eb1f89835ae93d937e0ea38ad59ae1ef5f7bc534d424d8c91e26374c9cf9b7f6d22845c7fa1ee270be1bd21 with token id 643 found, validating"}

---

Here is the Console:

[charleslcso]

It works-for-me in the classic web UI.

Please try logging in to the server from a browser and doing the same sot of thing - rename a folder to include an emoji in the name, then rename to remove the emoji from the name. And confirm if that works.

@phil-davis Can you give me your hardware configuration? I want to see if I can follow your setup and check the issue.

[charleslcso]

Installed OC using image on Intel hardware. All errors are exactly the same.

PostgreSQL database for OC is in UTF-8 format. File/directory names are stored properly.

OC just won't work in application level; always 500 when syncing files/directories with UTF-8 chars.

I've given up on OC.

[phil-davis]

@charleslcso sorry that I can't be of much help. I run this in my local development environment to test. I just went to my hosted oC10 storage and it works fine there. Send me a message at https://www.jankaritech.com/#contact and I can give you a temporary public link where you can try emojis in file and folder names and confirm that it works.

[charleslcso]

@charleslcso sorry that I can't be of much help. I run this in my local development environment to test. I just went to my hosted oC10 storage and it works fine there. Send me a message at https://www.jankaritech.com/#contact and I can give you a temporary public link where you can try emojis in file and folder names and confirm that it works.

Thanks @phil-davis.

Came across this lately. Interesting that someone faced some UTF-8 issue(s), 8 years ago Issue 12112.

[metawops]

Any news on this? I just came across this problem, too. Running ownCloud (10.10.0) on a Raspberry Pi 4 (Raspberry Pi OS buster). Here are a few logfile lines where writing failed. And, yes, all these files start with an emoji in their filenames.

{"reqId":"ebd95b9b-8c95-4f94-87a1-241525dfc7f2","level":3,"time":"2022-09-25T09:17:49+00:00","remoteAddr":"192.168.0.75","user":"stefan","app":"PHP","method":"PUT","url":"\/remote.php\/dav\/files\/stefan\/StefVault%20-%20Backup%202022-09-25\/%F0%9F%8F%A0%20Start%20Here.md","message":"fopen(\/media\/ownclouddrive\/\/stefan\/files\/StefVault - Backup 2022-09-25\/\ud83c\udfe0 Start Here.md.ocTransferId54376671.part): failed to open stream: Invalid argument at \/var\/www\/owncloud\/lib\/private\/Files\/Storage\/Local.php#298"}
{"reqId":"ebd95b9b-8c95-4f94-87a1-241525dfc7f2","level":3,"time":"2022-09-25T09:17:49+00:00","remoteAddr":"192.168.0.75","user":"stefan","app":"webdav","method":"PUT","url":"\/remote.php\/dav\/files\/stefan\/StefVault%20-%20Backup%202022-09-25\/%F0%9F%8F%A0%20Start%20Here.md","message":"\\OC\\Files\\Filesystem::fopen() failed"}
{"reqId":"ebd95b9b-8c95-4f94-87a1-241525dfc7f2","level":2,"time":"2022-09-25T09:17:49+00:00","remoteAddr":"192.168.0.75","user":"stefan","app":"core","method":"PUT","url":"\/remote.php\/dav\/files\/stefan\/StefVault%20-%20Backup%202022-09-25\/%F0%9F%8F%A0%20Start%20Here.md","message":"ignoring lock release with type 1 for files\/3a35c6a3b54d5b4768d38fe75eaebd16. Lock hasn't been acquired before"}
{"reqId":"ebd95b9b-8c95-4f94-87a1-241525dfc7f2","level":4,"time":"2022-09-25T09:17:49+00:00","remoteAddr":"192.168.0.75","user":"stefan","app":"webdav","method":"PUT","url":"\/remote.php\/dav\/files\/stefan\/StefVault%20-%20Backup%202022-09-25\/%F0%9F%8F%A0%20Start%20Here.md","message":"Exception: HTTP\/1.1 500 Could not write file contents: {\"Exception\":\"Sabre\\\\DAV\\\\Exception\",\"Message\":\"Could not write file contents\",\"Code\":0,\"Trace\":\"#0 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/Directory.php(173): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\File->put()\\n#1 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(1098): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\Directory->createFile()\\n#2 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(504): Sabre\\\\DAV\\\\Server->createFile()\\n#3 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/event\\\/lib\\\/WildcardEmitterTrait.php(89): Sabre\\\\DAV\\\\CorePlugin->httpPut()\\n#4 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(472): Sabre\\\\DAV\\\\Server->emit()\\n#5 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(253): Sabre\\\\DAV\\\\Server->invokeMethod()\\n#6 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/lib\\\/Server.php(349): Sabre\\\\DAV\\\\Server->start()\\n#7 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/appinfo\\\/v2\\\/remote.php(31): OCA\\\\DAV\\\\Server->exec()\\n#8 \\\/var\\\/www\\\/owncloud\\\/remote.php(165): require_once('\\\/var\\\/www\\\/ownclo...')\\n#9 {main}\",\"File\":\"\\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/File.php\",\"Line\":205}"}
{"reqId":"6a15a3fa-2291-40f8-bf80-395b3a412a49","level":3,"time":"2022-09-25T09:17:49+00:00","remoteAddr":"192.168.0.75","user":"stefan","app":"PHP","method":"PUT","url":"\/remote.php\/dav\/files\/stefan\/StefVault%20-%20Backup%202022-09-25\/%F0%9F%93%8B%20Kanban%20Board%20Test.md","message":"fopen(\/media\/ownclouddrive\/\/stefan\/files\/StefVault - Backup 2022-09-25\/\ud83d\udccb Kanban Board Test.md.ocTransferId12420043.part): failed to open stream: Invalid argument at \/var\/www\/owncloud\/lib\/private\/Files\/Storage\/Local.php#298"}
{"reqId":"6a15a3fa-2291-40f8-bf80-395b3a412a49","level":3,"time":"2022-09-25T09:17:49+00:00","remoteAddr":"192.168.0.75","user":"stefan","app":"webdav","method":"PUT","url":"\/remote.php\/dav\/files\/stefan\/StefVault%20-%20Backup%202022-09-25\/%F0%9F%93%8B%20Kanban%20Board%20Test.md","message":"\\OC\\Files\\Filesystem::fopen() failed"}
{"reqId":"6a15a3fa-2291-40f8-bf80-395b3a412a49","level":2,"time":"2022-09-25T09:17:49+00:00","remoteAddr":"192.168.0.75","user":"stefan","app":"core","method":"PUT","url":"\/remote.php\/dav\/files\/stefan\/StefVault%20-%20Backup%202022-09-25\/%F0%9F%93%8B%20Kanban%20Board%20Test.md","message":"ignoring lock release with type 1 for files\/31446d2687cb765a47c6b6b90a7c5332. Lock hasn't been acquired before"}
{"reqId":"6a15a3fa-2291-40f8-bf80-395b3a412a49","level":4,"time":"2022-09-25T09:17:49+00:00","remoteAddr":"192.168.0.75","user":"stefan","app":"webdav","method":"PUT","url":"\/remote.php\/dav\/files\/stefan\/StefVault%20-%20Backup%202022-09-25\/%F0%9F%93%8B%20Kanban%20Board%20Test.md","message":"Exception: HTTP\/1.1 500 Could not write file contents: {\"Exception\":\"Sabre\\\\DAV\\\\Exception\",\"Message\":\"Could not write file contents\",\"Code\":0,\"Trace\":\"#0 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/Directory.php(173): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\File->put()\\n#1 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(1098): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\Directory->createFile()\\n#2 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(504): Sabre\\\\DAV\\\\Server->createFile()\\n#3 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/event\\\/lib\\\/WildcardEmitterTrait.php(89): Sabre\\\\DAV\\\\CorePlugin->httpPut()\\n#4 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(472): Sabre\\\\DAV\\\\Server->emit()\\n#5 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(253): Sabre\\\\DAV\\\\Server->invokeMethod()\\n#6 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/lib\\\/Server.php(349): Sabre\\\\DAV\\\\Server->start()\\n#7 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/appinfo\\\/v2\\\/remote.php(31): OCA\\\\DAV\\\\Server->exec()\\n#8 \\\/var\\\/www\\\/owncloud\\\/remote.php(165): require_once('\\\/var\\\/www\\\/ownclo...')\\n#9 {main}\",\"File\":\"\\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/File.php\",\"Line\":205}"}
{"reqId":"f7f232f2-330e-4e03-affd-c568ef27d85b","level":3,"time":"2022-09-25T09:17:50+00:00","remoteAddr":"192.168.0.75","user":"stefan","app":"PHP","method":"PUT","url":"\/remote.php\/dav\/files\/stefan\/StefVault%20-%20Backup%202022-09-25\/%F0%9F%93%BA%20YouTube%20Video%20Ideen.md","message":"fopen(\/media\/ownclouddrive\/\/stefan\/files\/StefVault - Backup 2022-09-25\/\ud83d\udcfa YouTube Video Ideen.md.ocTransferId152643597.part): failed to open stream: Invalid argument at \/var\/www\/owncloud\/lib\/private\/Files\/Storage\/Local.php#298"}
{"reqId":"f7f232f2-330e-4e03-affd-c568ef27d85b","level":3,"time":"2022-09-25T09:17:50+00:00","remoteAddr":"192.168.0.75","user":"stefan","app":"webdav","method":"PUT","url":"\/remote.php\/dav\/files\/stefan\/StefVault%20-%20Backup%202022-09-25\/%F0%9F%93%BA%20YouTube%20Video%20Ideen.md","message":"\\OC\\Files\\Filesystem::fopen() failed"}
{"reqId":"f7f232f2-330e-4e03-affd-c568ef27d85b","level":2,"time":"2022-09-25T09:17:50+00:00","remoteAddr":"192.168.0.75","user":"stefan","app":"core","method":"PUT","url":"\/remote.php\/dav\/files\/stefan\/StefVault%20-%20Backup%202022-09-25\/%F0%9F%93%BA%20YouTube%20Video%20Ideen.md","message":"ignoring lock release with type 1 for files\/98f7dc4af96b608d227921e94ea03ad7. Lock hasn't been acquired before"}
{"reqId":"f7f232f2-330e-4e03-affd-c568ef27d85b","level":4,"time":"2022-09-25T09:17:50+00:00","remoteAddr":"192.168.0.75","user":"stefan","app":"webdav","method":"PUT","url":"\/remote.php\/dav\/files\/stefan\/StefVault%20-%20Backup%202022-09-25\/%F0%9F%93%BA%20YouTube%20Video%20Ideen.md","message":"Exception: HTTP\/1.1 500 Could not write file contents: {\"Exception\":\"Sabre\\\\DAV\\\\Exception\",\"Message\":\"Could not write file contents\",\"Code\":0,\"Trace\":\"#0 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/Directory.php(173): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\File->put()\\n#1 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(1098): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\Directory->createFile()\\n#2 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(504): Sabre\\\\DAV\\\\Server->createFile()\\n#3 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/event\\\/lib\\\/WildcardEmitterTrait.php(89): Sabre\\\\DAV\\\\CorePlugin->httpPut()\\n#4 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(472): Sabre\\\\DAV\\\\Server->emit()\\n#5 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(253): Sabre\\\\DAV\\\\Server->invokeMethod()\\n#6 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/lib\\\/Server.php(349): Sabre\\\\DAV\\\\Server->start()\\n#7 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/appinfo\\\/v2\\\/remote.php(31): OCA\\\\DAV\\\\Server->exec()\\n#8 \\\/var\\\/www\\\/owncloud\\\/remote.php(165): require_once('\\\/var\\\/www\\\/ownclo...')\\n#9 {main}\",\"File\":\"\\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/File.php\",\"Line\":205}"}

The emojis in these three files / the filenames are:

• 🏠 Start Here.md
• 📋 Kanban Board Test.md
• 📺 YouTube Video Ideen.md

But I get errors for filenames with German Umlauts, too. 🤷‍♂️

[ho4ho]

Both have Raspberry, problem not by ownCloud but by PHP or MariaDB? Could be problem 32bit? Or could be problem by Apache: #37677 ?

[garrettboone]

I had a related issue recently where mysql was only partially set to utf8-mb4, but the mysql connection settings were utf8-mb3. Once changed, and even after running the OC db conversion to utf8, I still needed to have the user change the name of the field (in this case it would be the folder name) to have no emoji, because it was sending over mb3-type which could not be accepted. Once they did that, they were able to change back to emoji and it worked fine. Maybe this will help.

[charleslcso]

@ho4ho, I have the same issue, but I'm running on Nginx and PostgreSQL. Also, I tried on both Intel and Pi. Same issue. Please see my posts above.

Just for fun, I installed the latest Nextcloud and tried the same thing; same errors... let's move.