hass-music-assistant
hass-music-assistant copied to clipboard
music-assistant
MA breaks recorder : 'charmap' codec can't encode character '\u0151'
What version of Music Assistant has the issue?
2022.8.2
The problem
please see: https://github.com/home-assistant/core/issues/76436#issuecomment-1207825100
apparently Music assistant injects an undecodable character '\u0151'
this is reported in HA core for recorder as an issue for safeguarding recorder integrity. But, as it stems from Music Assistant (after updating to 2022.8.2), it should probably be looked at here too ;-)
How to reproduce
Not sure, Ive done nothing really to any config, and only found out when trying to read another file in my HA system
Relevant log output
2022-08-08 09:36:08.120 ERROR (Recorder) [homeassistant.components.recorder.core] Error while processing event EventTask(event=<Event mass_event[L]: type=media_item_updated, object_id=database://album/17, data=item_id=17, provider=database, name=Vivaldi: Argippo, provider_ids=[{'item_id': '13WU1MoeIkk5FTKXhpNcH5', 'prov_type': 'spotify', 'prov_id': 'spotify_ainhdb', 'available': True, 'quality': 2, 'details': None, 'url': 'https://open.spotify.com/album/13WU1MoeIkk5FTKXhpNcH5'}], metadata=description=None, review=None, explicit=None, images=[{'type': 'thumb', 'url': 'https://i.scdn.co/image/ab67616d0000b273edb86a009276b6d94ab570b1', 'is_file': False}], genres=None, mood=None, style=None, copyright=2019 Naïve, a Label of Believe Group, lyrics=None, ean=None, label=Naïve, a Label of Believe Group, links=None, performers=None, preview=None, replaygain=None, popularity=None, last_refresh=None, checksum=None, in_library=True, media_type=album, sort_name=vivaldi: argippo, uri=database://album/17, timestamp=1659944168, version=, year=2020, artists=[{'item_id': '52', 'provider': 'database', 'name': 'Europa Galante', 'provider_ids': [], 'metadata': {'description': None, 'review': None, 'explicit': None, 'images': None, 'genres': None, 'mood': None, 'style': None, 'copyright': None, 'lyrics': None, 'ean': None, 'label': None, 'links': None, 'performers': None, 'preview': None, 'replaygain': None, 'popularity': None, 'last_refresh': None, 'checksum': None}, 'in_library': False, 'media_type': 'artist', 'sort_name': 'europa galante', 'uri': 'database://artist/52', 'timestamp': 0, 'musicbrainz_id': None}, {'item_id': '51', 'provider': 'database', 'name': 'Antonio Vivaldi', 'provider_ids': [], 'metadata': {'description': None, 'review': None, 'explicit': None, 'images': None, 'genres': None, 'mood': None, 'style': None, 'copyright': None, 'lyrics': None, 'ean': None, 'label': None, 'links': None, 'performers': None, 'preview': None, 'replaygain': None, 'popularity': None, 'last_refresh': None, 'checksum': None}, 'in_library': False, 'media_type': 'artist', 'sort_name': 'antonio vivaldi', 'uri': 'database://artist/51', 'timestamp': 0, 'musicbrainz_id': None}, {'item_id': '53', 'provider': 'database', 'name': 'Fabio Biondi', 'provider_ids': [], 'metadata': {'description': None, 'review': None, 'explicit': None, 'images': None, 'genres': None, 'mood': None, 'style': None, 'copyright': None, 'lyrics': None, 'ean': None, 'label': None, 'links': None, 'performers': None, 'preview': None, 'replaygain': None, 'popularity': None, 'last_refresh': None, 'checksum': None}, 'in_library': False, 'media_type': 'artist', 'sort_name': 'fabio biondi', 'uri': 'database://artist/53', 'timestamp': 0, 'musicbrainz_id': None}, {'item_id': '54', 'provider': 'database', 'name': 'Emőke Baráth', 'provider_ids': [], 'metadata': {'description': None, 'review': None, 'explicit': None, 'images': None, 'genres': None, 'mood': None, 'style': None, 'copyright': None, 'lyrics': None, 'ean': None, 'label': None, 'links': None, 'performers': None, 'preview': None, 'replaygain': None, 'popularity': None, 'last_refresh': None, 'checksum': None}, 'in_library': False, 'media_type': 'artist', 'sort_name': 'emőke baráth', 'uri': 'database://artist/54', 'timestamp': 0, 'musicbrainz_id': None}], album_type=album, upc=3616408945965, musicbrainz_id=None>): 'charmap' codec can't encode character '\u0151' in position 2512: character maps to <undefined>
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/recorder/core.py", line 632, in _run_event_loop
self._process_one_task_or_recover(task)
File "/usr/src/homeassistant/homeassistant/components/recorder/core.py", line 645, in _process_one_task_or_recover
return task.run(self)
File "/usr/src/homeassistant/homeassistant/components/recorder/tasks.py", line 211, in run
instance._process_one_event(self.event)
File "/usr/src/homeassistant/homeassistant/components/recorder/core.py", line 743, in _process_one_event
self._process_non_state_changed_event_into_session(event)
File "/usr/src/homeassistant/homeassistant/components/recorder/core.py", line 808, in _process_non_state_changed_event_into_session
if data_id := self._find_shared_data_in_db(data_hash, shared_data):
File "/usr/src/homeassistant/homeassistant/components/recorder/core.py", line 778, in _find_shared_data_in_db
if data_id := self.event_session.execute(
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 1712, in execute
result = conn._execute_20(statement, params or {}, execution_options)
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/lambdas.py", line 516, in _execute_on_connection
return connection._execute_clauseelement(
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement
ret = self._execute_context(
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context
self._handle_dbapi_exception(
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2047, in _handle_dbapi_exception
util.raise_(exc_info[1], with_traceback=exc_info[2])
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 208, in raise_
raise exception
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context
self.dialect.do_execute(
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
cursor.execute(statement, parameters)
File "/usr/local/lib/python3.10/site-packages/MySQLdb/cursors.py", line 199, in execute
args = tuple(map(db.literal, args))
File "/usr/local/lib/python3.10/site-packages/MySQLdb/connections.py", line 275, in literal
s = self.string_literal(o.encode(self.encoding))
File "/usr/local/lib/python3.10/encodings/cp1252.py", line 12, in encode
return codecs.charmap_encode(input,errors,encoding_table)
UnicodeEncodeError: 'charmap' codec can't encode character '\u0151' in position 2512: character maps to <undefined>
Additional information
No response
What version of Home Assistant Core are your running
2022.8.2
What type of installation are you running?
Home Assistant OS
On what type of hardware are you running?
Generic x86-64 (e.g. Intel NUC)
First of all we should probably leave the "mass events" out of the recorder in the first place because that is just a waste of resources. Secondly, the data is nothing fancy and just python unicode from utf-8 so I'm a bit surprised that the recorder code chokes on it.
tbh, I was indeed surprised to see those being recorded indeed, because I have a specific include policy.... and have explicitly excluded most events from being recorded. what is the event name for Music assistant to add to that list? would it simply be:
mass_event
to add to:
event_types:
- automation_triggered
- call_service
- component_loaded
- feedreader
- homeassistant_start
- homeassistant_stop
- logbook_entry
- platform_discovered
- script_started
- service_executed
- service_registered
- service_removed
# - system_log_event #needed for automations based in system_log_event?
- timer_out_of_sync
# extra event_types
- core_config_updated
- delayed_homeassistant_start
- device_registry_updated
# - device_tracker_new_device
# - entity_registry_updated
- event_mqtt_reloaded
- event_template_reloaded
- feedreader
# - google_assistant_command
- homeassistant_close
- homeassistant_final_write
- hue_event
- ios.notification_action_fired
# - life360_update_overdue
# - life360_update_restored
- lovelace_updated
# - mobile_app_notification_action
- panels_updated
- persistent_notifications_updated
# dont exclude state_changed events, or your includes won't be recorded.....
# - state_changed
- themes_updated
- user_removed
?
thx, added, and restarted....
What I dont get yet, is why reading the file it was trying to read, this occurred and blowup home assistant with a huuuuge log. It didnt involve any event by MA, and simply tries to read a text file as reported in the HA issue.
Yes, I am starting to believe that is was...
Ive just retried that, and the response is immediate, and the text file is displayed correctly.
while playing there's a mass_event every second so chances are big you will catch this event hence the coincidence.
hmmm, I wasnt playing anything, but maybe it was still updating the DB, it was seconds after startup?
Anyways, excluding the mass_event is now done, but will this actually do that 100%, or should MA code prevent it from being added completely
I did not even realize that the events are also written to the db/recorder by default (why????) and I don't think there's a way for an integration to exclude by default.
mass_event can be fired very often because mass is entirely event driven, for example when starting playback or when scanning your library with a lot of changes. It can't do any real harm or something but it's just a waste of storage to save these events in the db.
are you saying certain things now wont work, because of my excluding the mass_events from recorder?
btw, remember we discussed adding a NVME m.2 memory card to the Mele mini pc? As it happens, I finally moved the data yesterday. Which was a 2 minute job. Seems to have no speed effect though, so thats a bit disappointing.
Only mentioning here because it might have caused some recorder trouble? (which would be surprising really, because there's no indication at all that would be the case. )
are you saying certain things now wont work, because of my excluding the mass_events from recorder? No, not at all, you can just leave them out the recorder.
btw, remember we discussed adding a NVME m.2 memory card to the Mele mini pc? As it happens, I finally moved the data yesterday. Which was a 2 minute job. Seems to have no speed effect though, so thats a bit disappointing.
Ah ok, great to hear that was so easy. I don't think you can get it much faster as it already is blazing fast as-is. Maybe the db benefits a bit when it is stored on SSD but with the huge amount of memory, most important is already cached in memory.
Only mentioning here because it might have caused some recorder trouble? (which would be surprising really, because there's no indication at all that would be the case. )
No but maybe a small corruption of copying the database which I assume is mariadb ?
yep, mariadb. will see if I can find other DB related settings, maybe Ill purge it manually once to clean things up. though all graphs I have in HA are showing as they should, so in fact, no need from that pov.
Ah ok, great to hear that was so easy. I don't think you can get it much faster as it already is blazing fast as-is. Maybe the db benefits a bit when it is stored on SSD but with the huge amount of memory, most important is already cached in memory.
guess I might a just as well restore it to the internal sda then.. didnt yet measure the power it consumes now, and if there's a difference. might be just that bit more efficient. only wondering if we can go back in storage space with the move data-disk, recall that being an issue in the past
Hi @Mariusthvdb . Looking at your core issue it seems the problem lies not with MA (mariadb?) If so can we close this?
