files_texteditor
files_texteditor copied to clipboard
nextcloud
[Bug]: Files displays broken encoding when ⚰ is added to Markdown file
⚠️ This issue respects the following points: ⚠️
- [X] This is a bug, not a question or a configuration/webserver/proxy issue.
- [X] This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
- [X] Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
- [X] I agree to follow Nextcloud's Code of Conduct.
Bug description
After upgrading nextcloud to 26.0.5, I noticed that some Markdown files are shown with broken encoding in the editor. They were fine in many nextcloud versions up to 23.0.12. I updated in steps to 26.0.5. I also updated to 27.0.2, but the bug persists.
All files are encoded in UTF-8. I've tracked it down to: when I add a character ⚰ (U+26B0) it breaks. With äöü in the file, it displays correctly. With äöü⚰ in the file, it displays as äöü⚰ The files on the filesystem are fine. It's a problem with the nextcloud files editor.
I've used docker images nextcloud:23-fpm .. nextcloud:27-fpm
Steps to reproduce
- In Files, create a test.md file
- Click it to open it in the editor
- Add
test äöü - Close the file
- Open it again: it displays correctly
- Add
⚰. Still looking fine - Close the file
- Open it again: it displays
test äöü ⚰
Expected behavior
The file show display correctly as it did in older nextcloud versions.
Installation method
Community Docker image
Nextcloud Server version
27
Operating system
Debian/Ubuntu
PHP engine version
Other
Web server
Nginx
Database engine version
MariaDB
Is this bug present after an update or on a fresh install?
Upgraded to a MAJOR version (ex. 22 to 23)
Are you using the Nextcloud Server Encryption module?
None
What user-backends are you using?
- [X] Default user-backend (database)
- [ ] LDAP/ Active Directory
- [ ] SSO - SAML
- [ ] Other
Configuration report
{
"system": {
"instanceid": "***REMOVED SENSITIVE VALUE***",
"passwordsalt": "***REMOVED SENSITIVE VALUE***",
"secret": "***REMOVED SENSITIVE VALUE***",
"trusted_domains": [
"nextcloud.timesink.de",
"nc.timesink.de"
],
"datadirectory": "***REMOVED SENSITIVE VALUE***",
"overwrite.cli.url": "https:\/\/nextcloud.timesink.de",
"dbtype": "mysql",
"version": "27.0.2.1",
"dbname": "***REMOVED SENSITIVE VALUE***",
"dbhost": "***REMOVED SENSITIVE VALUE***",
"dbport": "",
"dbtableprefix": "oc_",
"dbuser": "***REMOVED SENSITIVE VALUE***",
"dbpassword": "***REMOVED SENSITIVE VALUE***",
"logtimezone": "UTC",
"instanceid": "***REMOVED SENSITIVE VALUE***", [0/28]
"passwordsalt": "***REMOVED SENSITIVE VALUE***",
"secret": "***REMOVED SENSITIVE VALUE***",
"trusted_domains": [
"nextcloud.timesink.de",
"nc.timesink.de"
],
"datadirectory": "***REMOVED SENSITIVE VALUE***",
"overwrite.cli.url": "https:\/\/nextcloud.timesink.de",
"dbtype": "mysql",
"version": "27.0.2.1",
"dbname": "***REMOVED SENSITIVE VALUE***",
"dbhost": "***REMOVED SENSITIVE VALUE***",
"dbport": "",
"dbtableprefix": "oc_",
"dbuser": "***REMOVED SENSITIVE VALUE***",
"dbpassword": "***REMOVED SENSITIVE VALUE***",
"logtimezone": "UTC",
"installed": true,
"maintenance": false,
"theme": "",
"loglevel": 2,
"mysql.utf8mb4": true,
"memcache.local": "\\OC\\Memcache\\Redis",
"memcache.distributed": "\\OC\\Memcache\\Redis",
"memcache.locking": "\\OC\\Memcache\\Redis",
"redis": {
"host": "***REMOVED SENSITIVE VALUE***",
"port": 6379
},
"mail_smtpmode": "smtp",
"mail_smtpsecure": "ssl",
"mail_sendmailmode": "smtp",
"mail_from_address": "***REMOVED SENSITIVE VALUE***",
"mail_domain": "***REMOVED SENSITIVE VALUE***",
"mail_smtpauthtype": "PLAIN",
"mail_smtpauth": 1,
"mail_smtphost": "***REMOVED SENSITIVE VALUE***",
"mail_smtpport": "465",
"mail_smtpname": "***REMOVED SENSITIVE VALUE***",
"mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
"updater.release.channel": "stable",
"default_phone_region": "DE"
}
}
List of activated Apps
Enabled:
- activity: 2.19.0
- bookmarks: 13.0.1
- bruteforcesettings: 2.7.0
- calendar: 4.4.4
- circles: 27.0.1
- cloud_federation_api: 1.10.0
- comments: 1.17.0
- contacts: 5.3.2
- contactsinteraction: 1.8.0
- cookbook: 0.10.2
- dashboard: 7.7.0
- dav: 1.27.0
- federatedfilesharing: 1.17.0
- federation: 1.17.0
- files: 1.22.0
- files_accesscontrol: 1.17.0
- files: 1.22.0 [0/89]
- files_accesscontrol: 1.17.0
- files_markdown: 2.4.1
- files_pdfviewer: 2.8.0
- files_rightclick: 1.6.0
- files_sharing: 1.19.0
- files_texteditor: 2.15.1
- files_trashbin: 1.17.0
- files_versions: 1.20.0
- firstrunwizard: 2.16.0
- gpoddersync: 3.8.1
- logreader: 2.12.0
- lookup_server_connector: 1.15.0
- nextcloud_announcements: 1.16.0
- notes: 4.8.1
- notifications: 2.15.0
- oauth2: 1.15.1
- password_policy: 1.17.0
- photos: 2.3.0
- privacy: 1.11.0
- provisioning_api: 1.17.0
- recommendations: 1.6.0
- related_resources: 1.2.0
- serverinfo: 1.17.0
- settings: 1.9.0
- sharebymail: 1.17.0
- support: 1.10.0
- survey_client: 1.15.0
- systemtags: 1.17.0
- tasks: 0.15.0
- theming: 2.2.0
- twofactor_backupcodes: 1.16.0
- updatenotification: 1.17.0
- user_status: 1.7.0
- viewer: 2.1.0
- weather_status: 1.7.0
- workflowengine: 2.9.0
Disabled:
- admin_audit: 1.17.0
- encryption: 2.15.0
- files_external: 1.19.0
- suspicious_login: 5.0.0
- text: 3.8.0 (installed 2.0.0)
- twofactor_totp: 9.0.0
- user_ldap: 1.17.0
Nextcloud Signing status
No errors have been found.
Nextcloud Logs
No response
Additional info
No response
I cannot reproduce that, but haven't tried the community docker image yet. Any chance you can share a copy of such a file here as an attachment for checking the file encoding directly?
The file is fine, I thoroughly checked the encoding.
It is also displaying fine in the Nextcloud Android app.
The problem is in the dual pane Markdown editor that opens when I click the file in .../index.php/apps/files?dir=/Notizen/Medien&openfile=122800 which gets me to .../apps/files/?dir=/Notizen/Medien&fileid=88758#filestexteditor.
# cat vol/nextcloud/data/rj/files/Notizen/Medien/test.md
test äöü ⚰
# recode utf8..dump < vol/nextcloud/data/rj/files/Notizen/Medien/test.md
UCS2 Mne Description
0074 t latin small letter t
0065 e latin small letter e
0073 s latin small letter s
0074 t latin small letter t
0020 SP space
00E4 a: latin small letter a with diaeresis
00F6 o: latin small letter o with diaeresis
00FC u: latin small letter u with diaeresis
0020 SP space
26B0
Only when I edit the file and the editor saves it, then the encoding gets messed up. I removed the symbol that triggers the bug, but still:
# cat vol/nextcloud/data/rj/files/Notizen/Medien/test.md
test äöü
# recode utf8..dump < vol/nextcloud/data/rj/files/Notizen/Medien/test.md
UCS2 Mne Description
0074 t latin small letter t
0065 e latin small letter e
0073 s latin small letter s
0074 t latin small letter t
0020 SP space
00C3 A? latin capital letter a with tilde
00A4 Cu currency sign
00C3 A? latin capital letter a with tilde
00B6 PI pilcrow sign
00C3 A? latin capital letter a with tilde
00BC 14 vulgar fraction one quarter
I restored the file. Here is the base64-encoded content for your diagnostics:
# base64 vol/nextcloud/data/rj/files/Notizen/Medien/test.md
dGVzdCDDpMO2w7wg4pqwCg==
By the way, the bug is not triggered by every character beyond U+00FF. With the € U+20AC it works fine. Same with ✄ U+2704. But with ⚰ U+26B0 it breaks.
| Code point | Result |
|---|---|
| 20AC | fine |
| 26A5 | broken |
| 26AF | broken |
| 26B0 | broken |
| 26B1 | broken |
| 26B2 | broken |
| 26FF | fine |
| 2704 | fine |