PlexKodiConnect
PlexKodiConnect copied to clipboard
croneter
CoreElec: "'ascii' codec can't encode characters"
First, I wanted to say thank you for all the work you guys do, this is truly an amazing project and I love and use it every day of my life! You all are awesome! Additionally, I wanted to note that this is very low priority for me personally, as it doesn't impact me at all. I don't sync playlists or use playlists, and for those that do...
Here is 2 easy "work-arounds" to this issue
- Rename the playlist default name (remove the heart)
- You can disable playlist sync completely (if your like me and don't use them anyways)
I did the following troubleshooting
- [X] I did try to restart Kodi :-)
- [X] I checked the PKC Frequently Asked Questions on the PKC wiki
- [X] I did try to reset the Kodi database by going to
PKC Settings -> Advanced -> "Reset the database and optionally reset PlexKodiConnect"and then hitting YES, NO - [X] I did check the existing issues on Github
- [X] Finally a Fresh install of latest stable kodi.
Describe the bug
I believe the latest PMS (At least the one I upgraded to: Version 1.22.2.4282) added some weird heart emojis in the music smart playlists. When it does its sync by default plyalists are selected, and it throws and ascii errors complaining about it.
'ascii' codec can't encode characters in position 40-41: ordinal not in range(128)
I disabled the playlist sync and the issue didn't pop up anymore, so I was able to confirm it was the playlist
I then went and looked at the playlists (since I don't really use them, I wasn't expecting to have a custom setup here), and found the heart emoji/icon thing in the default music playlists. (image included)
To Reproduce
Steps to reproduce the behavior:
- Update plex to latest (or maybe even add the heart emoji to a playlist)
- Manually Sync PKC
Expected behavior
Skip over the character or at least not fail and allow it to sync the rest?
Here is a snippet of the actual error log:
2021-04-14 22:11:43.793 T:4896 INFO <general>: PLEX.playlists: Starting playlist full sync
2021-04-14 22:11:43.805 T:5010 INFO <general>: My Music: Scanning for music info using worker thread, operation took 00:00
2021-04-14 22:11:43.834 T:4726 WARNING <general>: CGUIMediaWindow::OnMessage - updating in progress
2021-04-14 22:11:44.026 T:5049 INFO <general>: PLEX.default: plugin.video.plexkodiconnect started
2021-04-14 22:11:44.043 T:4896 ERROR <general>: PLEX.sync: Exception encountered: 'ascii' codec can't encode characters in position 40-41: ordinal not in range(128)
2021-04-14 22:11:44.043 T:4896 ERROR <general>: PLEX.utils: Error encountered: - 'ascii' codec can't encode characters in position 40-41: ordinal not in range(128)
2021-04-14 22:11:44.050 T:4896 ERROR <general>: PLEX.utils: _____________________________________________________________
2021-04-14 22:11:44.050 T:4896 ERROR <general>: PLEX.utils: Traceback (most recent call last):
2021-04-14 22:11:44.051 T:4896 ERROR <general>: PLEX.utils: File "/storage/.kodi/addons/plugin.video.plexkodiconnect/resources/lib/library_sync/common.py", line 31, in run
2021-04-14 22:11:44.051 T:4896 ERROR <general>: PLEX.utils: self._run()
2021-04-14 22:11:44.051 T:4896 ERROR <general>: PLEX.utils: File "/storage/.kodi/addons/plugin.video.plexkodiconnect/resources/lib/utils.py", line 902, in wrapper
2021-04-14 22:11:44.051 T:4896 ERROR <general>: PLEX.utils: result = func(*args, **kwargs)
2021-04-14 22:11:44.051 T:4896 ERROR <general>: PLEX.utils: File "/storage/.kodi/addons/plugin.video.plexkodiconnect/resources/lib/library_sync/full_sync.py", line 308, in _run
2021-04-14 22:11:44.052 T:4896 ERROR <general>: PLEX.utils: self.full_library_sync()
2021-04-14 22:11:44.052 T:4896 ERROR <general>: PLEX.utils: File "/storage/.kodi/addons/plugin.video.plexkodiconnect/resources/lib/library_sync/full_sync.py", line 250, in full_library_sync
2021-04-14 22:11:44.052 T:4896 ERROR <general>: PLEX.utils: if not playlists.full_sync() or self.should_cancel():
2021-04-14 22:11:44.052 T:4896 ERROR <general>: PLEX.utils: File "/storage/.kodi/addons/plugin.video.plexkodiconnect/resources/lib/playlists/__init__.py", line 174, in full_sync
2021-04-14 22:11:44.053 T:4896 ERROR <general>: PLEX.utils: return _full_sync()
2021-04-14 22:11:44.053 T:4896 ERROR <general>: PLEX.utils: File "/storage/.kodi/addons/plugin.video.plexkodiconnect/resources/lib/playlists/__init__.py", line 200, in _full_sync
2021-04-14 22:11:44.053 T:4896 ERROR <general>: PLEX.utils: kodi_pl.create(api.plex_id)
2021-04-14 22:11:44.053 T:4896 ERROR <general>: PLEX.utils: File "/storage/.kodi/addons/plugin.video.plexkodiconnect/resources/lib/playlists/kodi_pl.py", line 69, in create
2021-04-14 22:11:44.054 T:4896 ERROR <general>: PLEX.utils: _write_playlist_to_file(playlist, xml_playlist)
2021-04-14 22:11:44.054 T:4896 ERROR <general>: PLEX.utils: File "/storage/.kodi/addons/plugin.video.plexkodiconnect/resources/lib/playlists/kodi_pl.py", line 124, in _write_playlist_to_file
2021-04-14 22:11:44.054 T:4896 ERROR <general>: PLEX.utils: with open(playlist.kodi_path, 'wb') as f:
2021-04-14 22:11:44.055 T:4896 ERROR <general>: PLEX.utils: UnicodeEncodeError: 'ascii' codec can't encode characters in position 40-41: ordinal not in range(128)
2021-04-14 22:11:44.055 T:4896 ERROR <general>: PLEX.utils: _____________________________________________________________
2021-04-14 22:11:44.083 T:4726 WARNING <general>: CGUIMediaWindow::OnMessage - updating in progress
2021-04-14 22:11:44.096 T:5010 INFO <general>: My Music: Scanning for music info using worker thread, operation took 00:00
2021-04-14 22:11:44.125 T:4726 WARNING <general>: CGUIMediaWindow::OnMessage - updating in progress
2021-04-14 22:11:44.126 T:5049 INFO <general>: PLEX.default: plugin.video.plexkodiconnect stopped
@croneter sorry, I forgot to click "comment", earlier!
I closed the issue, because I couldn't provide you a debug file anymore, since I deleted the ❤️ emoji, I couldn't add it back in to recreate the issue and give you a proper debug file. I was trying to type :heart into plex and it wouldn't populate, I then tried copying and pasting from slack and it didn't, I ended up looking at a bunch of ways to add them, and none of them worked.
However, I just tried to copy/paste it in from iMessage instead, AND IT WORKED! I believe you can also do the keyboard shortcut :heart on this window as well. So I Should be able to replicate it again! :D I'll add a debug file later tonight when I get off work!
Also, If it helps at all, I was looking more, and found that my 2nd plex server (for testing/staging) didn't have any playlists defined prior to the update, and it didn't create the new one with the ❤️ emoji. However effected serve previously had 2 music playlists prior to the PMS update, and it added the additional playlists automatically?
Or at a minimum added the ❤️ to the "Fresh ❤️" and "Recently Played ❤️" playlists. Which I don't remember creating these...
So to recap, it seems it might effect someone who has previously used music playlists in PMS, then updated PMS? And/Or if someone tried to copy/paste emoji's into Plex Playlist titles.
@croneter I took lunch early! Here is the log: cleaned-debug_kodi.log
I tried uploading it to pastebin.com, and hit a file size limit, so I attached it here. Search for ascii and see the error, and surprisingly you can actually search for ❤️ to the log, and it made it into the log file! Let me know if there is anything else I can do to help!
Should be fixed in PKC BETA version 3.1.2. Let me know here if this is not the case!
Maybe that's the solution:https://diewland.medium.com/python3-unicodeencodeerror-ascii-codec-can-t-encode-characters-in-position-0-9-ordinal-not-in-70ce922c552 export PYTHONIOENCODING=utf-8
No, unfortunately, this issue is a bit more complicated. This PKC function causes it right now:
with open(playlist.kodi_path, 'wb') as f:
f.write(text)
playlist.kodi_path is a str/unicode object. text is a bytes-object, encoded using utf-8 in your case (shows up in the log):
INFO <general>: PLEX.service: Playlist m3u encoding: utf-8
The Python standard function open is responsible to encode this str-path to a bytes-object in order for the opening-process to work. And that's where things go wrong. Python 3 should now use os.fsencode to automatically encode correctly. In your case, os.fsencode does NOT use utf-8 like we would expect it to, but uses ascii instead. This seems simply wrong.
I could fix this issue, but it will always be a really nasty workaround. I'd need to adapt every single open-function!
You're using CoreElec. The OS (=CoreElec / Linux) or the Python distribution is responsible for os.fsencode and sys.getfilesystemencoding to work correctly and e.g. return utf-8 instead of ascii. Kindly take this back to the CoreElec-team. Or stick to Kodi 18 for now (I do!). Or use the more stable LibreElec
Thanks for the reply, I found another thing, it's impossible to play movie with external subtitle, is this related to this issue what you mentioned?
Very likely as for external subtitles to work, I'd need to access the filesystem correctly.
I think on CoreELEC Locale addon from CoreELEC repository must be installed. Then system-wide encoding will be UTF-8 and not ascii.
So the issue is reproducible on LibreElec 10 also even after installing the Locale addon. Running the following commands on the OS returns UTF-8
sys.getdefaultencoding();
# utf-8
sys.getfilesystemencoding();
# utf-8
but in Kodi when plexKodiConnect initializes in variables.py
M3U_ENCODING = sys.getfilesystemencoding()
# ascii
So for a workaround, I just updated the code to explicitly use utf-8 encoding for subtitle path when opening it.
with open(path.encode("utf-8"), 'wb') as f:
f.write(response.content)
in file /storage/.kodi/addons/plugin.video.plexkodiconnect/resources/lib/media.py line 366
@staticmethod
def download_external_subtitles(url, filename, extension):
"""
One cannot pass the subtitle language for ListItems. Workaround; will
download the subtitle at url to the Kodi PKC directory in a temp dir
Returns the path to the downloaded subtitle or None
"""
path = path_ops.create_unique_path(v.EXTERNAL_SUBTITLE_TEMP_PATH,
filename,
extension)
response = DU().downloadUrl(url, return_response=True)
if not response.ok:
LOG.error('Could not temporarily download subtitle %s', url)
LOG.error('HTTP status: %s, message: %s',
response.status_code, response.text)
return
LOG.debug('Writing temp subtitle to %s', path)
with open(path.encode("utf-8"), 'wb') as f:
f.write(response.content)
return path
With this .encode("utf-8") I'm able to play everything like before.
Hello, I'm facing the exact same issue on LibreELEC 10, and I confirm @zingz0r's patch is working correctly ♥️
@Asphahyre I'd suggest to NOT mess with PKC source code but just install one additional add-on:
CoreELEC Locale addon from CoreELEC repository must be installed
@Asphahyre I'd suggest to NOT mess with PKC source code but just install one additional add-on:
CoreELEC Locale addon from CoreELEC repository must be installed
@croneter Thanks for the fast answer!
At this moment, rollbacking my change and installing Locale (ver. 10.0.0.102), from Team LibreELEC, and setting a system-wide UTF-8 locale, seems to have no effect (after reboot of course). Issue stil appears.
I will try with the one from CoreELEC repository then. Is it supposed to have a different behavior?
EDIT: I think the one I installed was the right one. I confirm I have the locale properly configured (sys.getfilesystemencoding(); returns utf-8).
I can't find the OpenELEC add-on repository anyway.
Can I provide more details?
Is there a solution for this problem?
LibreELEC with Locale installed returns the error below if I try to transcode and an external .SRT subtitle is present.
PLEX.utils: UnicodeEncodeError: 'ascii' codec can't encode character '\xea' in position 76: ordinal not in range(128)
@croneter
I have a RPi4 with LibreELEC, therefore "Locale" addon from the official LibreELEC repo was installed.
I assume that you don't really mean installing the Locale addon from CoreELEC (just checking)
@Asphahyre reported the same problem, the installation of Locale addon in a LibreELEC did not solve the issue.
New findings with my setup. RPi4 with LibreELEC 10.0.1 and Locale 10.0.0.102
No "UnicodeEncodeError: 'ascii' codec..." error when Kodi interface is set to English and a full reboot is done.
In fact if I later change Kodi's interface language from English to another one without making a full reboot it still works without errors.
@bruno-briner kindly activate DEBUG logging, reboot Kodi, replicate the bug and share your log file here. See https://github.com/croneter/PlexKodiConnect/wiki/How-to-Report-A-Bug
@croneter, thanks for the support. I hope the below logs help finding the root cause of the issue.
-
Kodi Interface in EN-GB and Locale in EN_GB => No error transcoding a 4k Movie with external subtitles in .SRT kodi_EN-GB_No-Errors.log
-
Kodi interface in PT-BR and Locale in PT_BR => "UnicodeEncodeError: 'ascii' codec can't encode character '\xea'" kodi_PT-BR_Errors.log
-
Kodi interface in PT-BR (but changed from EN-GB without a reboot) and Locale in PT_BR => No error transcoding a 4k Movie with external subtitles in .SRT kodi_PT-BR_from_EN-GB_No_Errors.log
So if I don't want this error popping up, a quick fix is to remove the ❤️ in the playlist name. But if I can live with the error, it doesn't seem that there's a quick fix around the corner, speaking months, correct?
