ytmusicapi
ytmusicapi copied to clipboard
sigma67
Error when querying get_home()
get_home() query no longer works
Hello, I noticed recently that the script I used to get music suggestions no longer works. The error message isn't very clear so I'm not sure what the issue might be but the script worked a few days ago and also returns results when using something like yt.search("Oasis Wonderwall")...
To Reproduce
Test Script using get_home ( doesn't work)
from ytmusicapi import YTMusic
import pprint
yt = YTMusic(oauth.json")
search_results = yt.get_home(30)
pprint.pprint(search_results)
Result
Traceback (most recent call last):
File "/home/user/.local/share/pipx/venvs/ytmusicapi/lib/python3.12/site-packages/ytmusicapi/navigation.py", line 107, in nav
root = root[k]
~~~~^^^
KeyError: 'watchEndpoint'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/user/.local/bin/ytmusicapi_home.py", line 14, in <module>
search_results = yt.get_home(30)
^^^^^^^^^^^^^^^
File "/home/user/.local/share/pipx/venvs/ytmusicapi/lib/python3.12/site-packages/ytmusicapi/mixins/browsing.py", line 118, in get_home
home.extend(parse_mixed_content(results))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/share/pipx/venvs/ytmusicapi/lib/python3.12/site-packages/ytmusicapi/parsers/browsing.py", line 27, in parse_mixed_content
content = parse_song(data)
^^^^^^^^^^^^^^^^
File "/home/user/.local/share/pipx/venvs/ytmusicapi/lib/python3.12/site-packages/ytmusicapi/parsers/browsing.py", line 86, in parse_song
"videoId": nav(result, NAVIGATION_VIDEO_ID),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/share/pipx/venvs/ytmusicapi/lib/python3.12/site-packages/ytmusicapi/navigation.py", line 111, in nav
raise type(e)(f"Unable to find '{k}' using path {items!r} on {root!r}, exception: {e}")
KeyError: "Unable to find 'watchEndpoint' using path ['navigationEndpoint', 'watchEndpoint', 'videoId'] on {'clickTrackingParams': 'CLECEKCzAhgAIhMIiZiNtpq6iQMVTgq3AB2ZTTys', 'browseEndpoint': {'browseEndpointContextSupportedConfigs': {'browseEndpointContextMusicConfig': {'pageType': 'MUSIC_PAGE_TYPE_ARTIST'}}}}, exception: 'watchEndpoint'"
Note that I just tried to generate oauth using another account and this appears to work. Also, I just modified navigation.py to print the content of the root variable and it does contain data, it even contains keys called watchEndpoint so I'm not 100% sure what the case of the issue is... could it be the some issue with the type of playlists (music Vs videos?)
Edit2: I tried using get_home(1) and it worked, then tried again and it no longer work. So perhaps an again issue with the content of the results?
Edit3: using timestamps, I was able to establish that the script does run for a few entries. However, when it lands on some it cannot find the 'watchEndpoint' key and crashing instead of skipping to the next. Is it possible to continue rather than crashing?
ok, I temporarily "solved it" by updating navigation.py from
try:
for k in items:
root = root[k]
except (KeyError, IndexError, TypeError) as e:
if none_if_absent:
return None
raise type(e)(f"Unable to find '{k}' using path {items!r} on {root!r}, exception: {e}")
return root
to
try:
for k in items:
root = root[k]
except (KeyError, IndexError, TypeError) as e:
if none_if_absent:
return None
raise type(e)(f"Unable to find '{k}' using path {items!r} on {root!r}, exception: {e}")
return root
but frankly, I still don't think that this shouldn't result in a crash, at best it should be a warning and skip?
Hi @Batwam
get_home is hard to debug because it is very user dependent. Could you provide reproduction instructions?
What did your homepage look like?
Hello, apologies for the delay as it was a bit tricky to troubleshoot. Responding here even though you closed it as I still believe that this is a bug. I tried to progressively increase the get_home(x) value to see at which section if would fail.
Turns out to be related to the "Listen Again" category with albums which I believe are manual uploads including tracks which I believe have been deleted... Not sure why they are still getting recommended but turns out that some (albums 2 and 3 below) are empty have a link which is 'https://music.youtube.com/channel/undefined' and lead to nowhere essentially.
log below where I'm printing root[k]. First artist works fine, second one crashes:
- text - Listen again
----------------
- musicTwoRowItemRenderer - {'thumbnailRenderer': {'musicThumbnailRenderer': {'thumbnail': {'thumbnails': [{'url': 'https://lh3.googleusercontent.com/a-/ALV-UjUbQGzZ2_DL08yAi4pQAupcyj2upWfEkLSFajn7LbQ8dDngP9y4cQ=w226-h226-l90-rj', 'width': 226, 'height': 226}, {'url': 'https://lh3.googleusercontent.com/a-/ALV-UjUbQGzZ2_DL08yAi4pQAupcyj2upWfEkLSFajn7LbQ8dDngP9y4cQ=w544-h544-l90-rj', 'width': 544, 'height': 544}]}, 'thumbnailCrop': 'MUSIC_THUMBNAIL_CROP_CIRCLE', 'thumbnailScale': 'MUSIC_THUMBNAIL_SCALE_ASPECT_FILL', 'trackingParams': 'CI8DEIS_AiITCPyb49Wmx4oDFbmg2AUdzocMmA=='}}, 'aspectRatio': 'MUSIC_TWO_ROW_ITEM_THUMBNAIL_ASPECT_RATIO_SQUARE', 'title': {'runs': [{'text': 'Preston Reed', 'navigationEndpoint': {'clickTrackingParams': 'CIkDEKCzAhgAIhMI_Jvj1abHigMVuaDYBR3OhwyY', 'browseEndpoint': {'browseId': 'UCvk7l9iOh4N_TwF0KFOmbQw', 'browseEndpointContextSupportedConfigs': {'browseEndpointContextMusicConfig': {'pageType': 'MUSIC_PAGE_TYPE_ARTIST'}}}}}]}, 'subtitle': {'runs': [{'text': '11.4K subscribers'}]}, 'navigationEndpoint': {'clickTrackingParams': 'CIkDEKCzAhgAIhMI_Jvj1abHigMVuaDYBR3OhwyY', 'browseEndpoint': {'browseId': 'UCvk7l9iOh4N_TwF0KFOmbQw', 'browseEndpointContextSupportedConfigs': {'browseEndpointContextMusicConfig': {'pageType': 'MUSIC_PAGE_TYPE_ARTIST'}}}}, 'trackingParams': 'CIkDEKCzAhgAIhMI_Jvj1abHigMVuaDYBR3OhwyY', 'menu': {'menuRenderer': {'items': [{'menuNavigationItemRenderer': {'text': {'runs': [{'text': 'Shuffle play'}]}, 'icon': {'iconType': 'MUSIC_SHUFFLE'}, 'navigationEndpoint': {'clickTrackingParams': 'CI4DEJrzBRgAIhMI_Jvj1abHigMVuaDYBR3OhwyY', 'watchPlaylistEndpoint': {'playlistId': 'RDAO8B2oAM_ijCcc991Z5qQCdA', 'params': 'wAEB8gECGAE%3D'}}, 'trackingParams': 'CI4DEJrzBRgAIhMI_Jvj1abHigMVuaDYBR3OhwyY'}}, {'menuNavigationItemRenderer': {'text': {'runs': [{'text': 'Start radio'}]}, 'icon': {'iconType': 'MIX'}, 'navigationEndpoint': {'clickTrackingParams': 'CI0DEJvzBRgBIhMI_Jvj1abHigMVuaDYBR3OhwyY', 'watchPlaylistEndpoint': {'playlistId': 'RDEM8B2oAM_ijCcc991Z5qQCdA', 'params': 'wAEB'}}, 'trackingParams': 'CI0DEJvzBRgBIhMI_Jvj1abHigMVuaDYBR3OhwyY'}}, {'toggleMenuServiceItemRenderer': {'defaultText': {'runs': [{'text': 'Subscribe'}]}, 'defaultIcon': {'iconType': 'SUBSCRIBE'}, 'defaultServiceEndpoint': {'clickTrackingParams': 'CIwDEKicBhgCIhMI_Jvj1abHigMVuaDYBR3OhwyY', 'subscribeEndpoint': {'channelIds': ['UCs61FMW9q9MIJXa154PbcYQ'], 'params': 'GAA%3D'}}, 'toggledText': {'runs': [{'text': 'Unsubscribe'}]}, 'toggledIcon': {'iconType': 'SUBSCRIBE'}, 'toggledServiceEndpoint': {'clickTrackingParams': 'CIwDEKicBhgCIhMI_Jvj1abHigMVuaDYBR3OhwyY', 'unsubscribeEndpoint': {'channelIds': ['UCs61FMW9q9MIJXa154PbcYQ'], 'params': 'GAA%3D'}}, 'trackingParams': 'CIwDEKicBhgCIhMI_Jvj1abHigMVuaDYBR3OhwyY'}}, {'menuNavigationItemRenderer': {'text': {'runs': [{'text': 'Share'}]}, 'icon': {'iconType': 'SHARE'}, 'navigationEndpoint': {'clickTrackingParams': 'CIsDEJH7BRgDIhMI_Jvj1abHigMVuaDYBR3OhwyY', 'shareEntityEndpoint': {'serializedShareEntity': 'GhhVQ3ZrN2w5aU9oNE5fVHdGMEtGT21iUXc%3D', 'sharePanelType': 'SHARE_PANEL_TYPE_UNIFIED_SHARE_PANEL'}}, 'trackingParams': 'CIsDEJH7BRgDIhMI_Jvj1abHigMVuaDYBR3OhwyY'}}], 'trackingParams': 'CIoDEKc7IhMI_Jvj1abHigMVuaDYBR3OhwyY', 'accessibility': {'accessibilityData': {'label': 'Action menu'}}}}}
----------------
- title - {'runs': [{'text': 'Preston Reed', 'navigationEndpoint': {'clickTrackingParams': 'CIkDEKCzAhgAIhMI_Jvj1abHigMVuaDYBR3OhwyY', 'browseEndpoint': {'browseId': 'UCvk7l9iOh4N_TwF0KFOmbQw', 'browseEndpointContextSupportedConfigs': {'browseEndpointContextMusicConfig': {'pageType': 'MUSIC_PAGE_TYPE_ARTIST'}}}}}]}
- runs - [{'text': 'Preston Reed', 'navigationEndpoint': {'clickTrackingParams': 'CIkDEKCzAhgAIhMI_Jvj1abHigMVuaDYBR3OhwyY', 'browseEndpoint': {'browseId': 'UCvk7l9iOh4N_TwF0KFOmbQw', 'browseEndpointContextSupportedConfigs': {'browseEndpointContextMusicConfig': {'pageType': 'MUSIC_PAGE_TYPE_ARTIST'}}}}}]
- 0 - {'text': 'Preston Reed', 'navigationEndpoint': {'clickTrackingParams': 'CIkDEKCzAhgAIhMI_Jvj1abHigMVuaDYBR3OhwyY', 'browseEndpoint': {'browseId': 'UCvk7l9iOh4N_TwF0KFOmbQw', 'browseEndpointContextSupportedConfigs': {'browseEndpointContextMusicConfig': {'pageType': 'MUSIC_PAGE_TYPE_ARTIST'}}}}}
- navigationEndpoint - {'clickTrackingParams': 'CIkDEKCzAhgAIhMI_Jvj1abHigMVuaDYBR3OhwyY', 'browseEndpoint': {'browseId': 'UCvk7l9iOh4N_TwF0KFOmbQw', 'browseEndpointContextSupportedConfigs': {'browseEndpointContextMusicConfig': {'pageType': 'MUSIC_PAGE_TYPE_ARTIST'}}}}
- browseEndpoint - {'browseId': 'UCvk7l9iOh4N_TwF0KFOmbQw', 'browseEndpointContextSupportedConfigs': {'browseEndpointContextMusicConfig': {'pageType': 'MUSIC_PAGE_TYPE_ARTIST'}}}
- browseEndpointContextSupportedConfigs - {'browseEndpointContextMusicConfig': {'pageType': 'MUSIC_PAGE_TYPE_ARTIST'}}
- browseEndpointContextMusicConfig - {'pageType': 'MUSIC_PAGE_TYPE_ARTIST'}
- pageType - MUSIC_PAGE_TYPE_ARTIST
----------------
- subtitle - {'runs': [{'text': '11.4K subscribers'}]}
- runs - [{'text': '11.4K subscribers'}]
- 0 - {'text': '11.4K subscribers'}
- text - 11.4K subscribers
----------------
- title - {'runs': [{'text': 'Preston Reed', 'navigationEndpoint': {'clickTrackingParams': 'CIkDEKCzAhgAIhMI_Jvj1abHigMVuaDYBR3OhwyY', 'browseEndpoint': {'browseId': 'UCvk7l9iOh4N_TwF0KFOmbQw', 'browseEndpointContextSupportedConfigs': {'browseEndpointContextMusicConfig': {'pageType': 'MUSIC_PAGE_TYPE_ARTIST'}}}}}]}
- runs - [{'text': 'Preston Reed', 'navigationEndpoint': {'clickTrackingParams': 'CIkDEKCzAhgAIhMI_Jvj1abHigMVuaDYBR3OhwyY', 'browseEndpoint': {'browseId': 'UCvk7l9iOh4N_TwF0KFOmbQw', 'browseEndpointContextSupportedConfigs': {'browseEndpointContextMusicConfig': {'pageType': 'MUSIC_PAGE_TYPE_ARTIST'}}}}}]
- 0 - {'text': 'Preston Reed', 'navigationEndpoint': {'clickTrackingParams': 'CIkDEKCzAhgAIhMI_Jvj1abHigMVuaDYBR3OhwyY', 'browseEndpoint': {'browseId': 'UCvk7l9iOh4N_TwF0KFOmbQw', 'browseEndpointContextSupportedConfigs': {'browseEndpointContextMusicConfig': {'pageType': 'MUSIC_PAGE_TYPE_ARTIST'}}}}}
- text - Preston Reed
----------------
- title - {'runs': [{'text': 'Preston Reed', 'navigationEndpoint': {'clickTrackingParams': 'CIkDEKCzAhgAIhMI_Jvj1abHigMVuaDYBR3OhwyY', 'browseEndpoint': {'browseId': 'UCvk7l9iOh4N_TwF0KFOmbQw', 'browseEndpointContextSupportedConfigs': {'browseEndpointContextMusicConfig': {'pageType': 'MUSIC_PAGE_TYPE_ARTIST'}}}}}]}
- runs - [{'text': 'Preston Reed', 'navigationEndpoint': {'clickTrackingParams': 'CIkDEKCzAhgAIhMI_Jvj1abHigMVuaDYBR3OhwyY', 'browseEndpoint': {'browseId': 'UCvk7l9iOh4N_TwF0KFOmbQw', 'browseEndpointContextSupportedConfigs': {'browseEndpointContextMusicConfig': {'pageType': 'MUSIC_PAGE_TYPE_ARTIST'}}}}}]
- 0 - {'text': 'Preston Reed', 'navigationEndpoint': {'clickTrackingParams': 'CIkDEKCzAhgAIhMI_Jvj1abHigMVuaDYBR3OhwyY', 'browseEndpoint': {'browseId': 'UCvk7l9iOh4N_TwF0KFOmbQw', 'browseEndpointContextSupportedConfigs': {'browseEndpointContextMusicConfig': {'pageType': 'MUSIC_PAGE_TYPE_ARTIST'}}}}}
- navigationEndpoint - {'clickTrackingParams': 'CIkDEKCzAhgAIhMI_Jvj1abHigMVuaDYBR3OhwyY', 'browseEndpoint': {'browseId': 'UCvk7l9iOh4N_TwF0KFOmbQw', 'browseEndpointContextSupportedConfigs': {'browseEndpointContextMusicConfig': {'pageType': 'MUSIC_PAGE_TYPE_ARTIST'}}}}
- browseEndpoint - {'browseId': 'UCvk7l9iOh4N_TwF0KFOmbQw', 'browseEndpointContextSupportedConfigs': {'browseEndpointContextMusicConfig': {'pageType': 'MUSIC_PAGE_TYPE_ARTIST'}}}
- browseId - UCvk7l9iOh4N_TwF0KFOmbQw
----------------
- thumbnailRenderer - {'musicThumbnailRenderer': {'thumbnail': {'thumbnails': [{'url': 'https://lh3.googleusercontent.com/a-/ALV-UjUbQGzZ2_DL08yAi4pQAupcyj2upWfEkLSFajn7LbQ8dDngP9y4cQ=w226-h226-l90-rj', 'width': 226, 'height': 226}, {'url': 'https://lh3.googleusercontent.com/a-/ALV-UjUbQGzZ2_DL08yAi4pQAupcyj2upWfEkLSFajn7LbQ8dDngP9y4cQ=w544-h544-l90-rj', 'width': 544, 'height': 544}]}, 'thumbnailCrop': 'MUSIC_THUMBNAIL_CROP_CIRCLE', 'thumbnailScale': 'MUSIC_THUMBNAIL_SCALE_ASPECT_FILL', 'trackingParams': 'CI8DEIS_AiITCPyb49Wmx4oDFbmg2AUdzocMmA=='}}
- musicThumbnailRenderer - {'thumbnail': {'thumbnails': [{'url': 'https://lh3.googleusercontent.com/a-/ALV-UjUbQGzZ2_DL08yAi4pQAupcyj2upWfEkLSFajn7LbQ8dDngP9y4cQ=w226-h226-l90-rj', 'width': 226, 'height': 226}, {'url': 'https://lh3.googleusercontent.com/a-/ALV-UjUbQGzZ2_DL08yAi4pQAupcyj2upWfEkLSFajn7LbQ8dDngP9y4cQ=w544-h544-l90-rj', 'width': 544, 'height': 544}]}, 'thumbnailCrop': 'MUSIC_THUMBNAIL_CROP_CIRCLE', 'thumbnailScale': 'MUSIC_THUMBNAIL_SCALE_ASPECT_FILL', 'trackingParams': 'CI8DEIS_AiITCPyb49Wmx4oDFbmg2AUdzocMmA=='}
- thumbnail - {'thumbnails': [{'url': 'https://lh3.googleusercontent.com/a-/ALV-UjUbQGzZ2_DL08yAi4pQAupcyj2upWfEkLSFajn7LbQ8dDngP9y4cQ=w226-h226-l90-rj', 'width': 226, 'height': 226}, {'url': 'https://lh3.googleusercontent.com/a-/ALV-UjUbQGzZ2_DL08yAi4pQAupcyj2upWfEkLSFajn7LbQ8dDngP9y4cQ=w544-h544-l90-rj', 'width': 544, 'height': 544}]}
- thumbnails - [{'url': 'https://lh3.googleusercontent.com/a-/ALV-UjUbQGzZ2_DL08yAi4pQAupcyj2upWfEkLSFajn7LbQ8dDngP9y4cQ=w226-h226-l90-rj', 'width': 226, 'height': 226}, {'url': 'https://lh3.googleusercontent.com/a-/ALV-UjUbQGzZ2_DL08yAi4pQAupcyj2upWfEkLSFajn7LbQ8dDngP9y4cQ=w544-h544-l90-rj', 'width': 544, 'height': 544}]
The part right after the one above is where is starts with what looks like partial data, then fails:
- musicTwoRowItemRenderer - {'thumbnailRenderer': {'musicThumbnailRenderer': {'thumbnail': {'thumbnails': [{'url': 'https://i9.ytimg.com/vi_locker/8Xio21X5hLQ/locker.png?sqp=-oaymwEICOIBEOIBIAA&rs=AMzJL3m7hQC8IRXFqMaDJaLjVf4Cw77_iw', 'width': 226, 'height': 226}, {'url': 'https://i9.ytimg.com/vi_locker/8Xio21X5hLQ/locker.png?sqp=-oaymwEICKAEEKAEIAA&rs=AMzJL3kZS8e-k-iYR7QVCtQnQLuiGj3Vvg', 'width': 544, 'height': 544}]}, 'thumbnailCrop': 'MUSIC_THUMBNAIL_CROP_CIRCLE', 'thumbnailScale': 'MUSIC_THUMBNAIL_SCALE_ASPECT_FILL', 'trackingParams': 'CIgDEIS_AiITCPyb49Wmx4oDFbmg2AUdzocMmA=='}}, 'aspectRatio': 'MUSIC_TWO_ROW_ITEM_THUMBNAIL_ASPECT_RATIO_SQUARE', 'title': {'runs': [{'text': 'Anthony M'}]}, 'subtitle': {}, 'navigationEndpoint': {'clickTrackingParams': 'CIcDEKCzAhgBIhMI_Jvj1abHigMVuaDYBR3OhwyY', 'browseEndpoint': {'browseEndpointContextSupportedConfigs': {'browseEndpointContextMusicConfig': {'pageType': 'MUSIC_PAGE_TYPE_ARTIST'}}}}, 'trackingParams': 'CIcDEKCzAhgBIhMI_Jvj1abHigMVuaDYBR3OhwyY'}
----------------
- title - {'runs': [{'text': 'Anthony M'}]}
- runs - [{'text': 'Anthony M'}]
- 0 - {'text': 'Anthony M'}
----------------
- navigationEndpoint - {'clickTrackingParams': 'CIcDEKCzAhgBIhMI_Jvj1abHigMVuaDYBR3OhwyY', 'browseEndpoint': {'browseEndpointContextSupportedConfigs': {'browseEndpointContextMusicConfig': {'pageType': 'MUSIC_PAGE_TYPE_ARTIST'}}}}
----------------
- title - {'runs': [{'text': 'Anthony M'}]}
- runs - [{'text': 'Anthony M'}]
- 0 - {'text': 'Anthony M'}
- text - Anthony M
----------------
- navigationEndpoint - {'clickTrackingParams': 'CIcDEKCzAhgBIhMI_Jvj1abHigMVuaDYBR3OhwyY', 'browseEndpoint': {'browseEndpointContextSupportedConfigs': {'browseEndpointContextMusicConfig': {'pageType': 'MUSIC_PAGE_TYPE_ARTIST'}}}}
as you can see, it's missing a bunch of fields and eventually crashes, maybe because it's missing ['browseEndpoint']['browseId'] within navigationEndpoint in particular?