KeyError: 'id' missing

[mxia9416]

When input a playlist url it will report KeyError 'id'', Metube works fine when input is a single video. In addition, I tried the same playlist url with newest release of yt-dlp(win11) fork provided in readme and it worked fine, so I create an issue ticket here. Log below:

DEBUG:ytdl:Processing as a video                                                 
ERROR:aiohttp.server:Error handling request                                      
Traceback (most recent call last):                                               
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_protocol.py", line 47
7, in _handle_request                                                            
    resp = await request_handler(request)                                                                              
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_app.py", line 559, in
 _handle                                                                         
    return await handler(request)                                                                                               
  File "/app/app/main.py", line 146, in add                                      
    status = await dqueue.add(url, quality, format, folder, custom_name_prefix, p
laylist_strict_mode, playlist_item_limit, auto_start)                                                
  File "/app/app/ytdl.py", line 337, in add                                      
    return await self.__add_entry(entry, quality, format, folder, custom_name_pre
fix, playlist_strict_mode, playlist_item_limit, auto_start, already)                      
  File "/app/app/ytdl.py", line 289, in __add_entry                              
    results.append(await self.__add_entry(etr, quality, format, folder, custom_na
me_prefix, playlist_strict_mode, playlist_item_limit, auto_start, already))      
  File "/app/app/ytdl.py", line 295, in __add_entry                              
    if not self.queue.exists(entry['id']):                                                            
KeyError: 'id'          

[PikuZheng]

Are you using the web ui to add the list or posting the list directly to the /add ?

[mxia9416]

I used webui, Metube itself is running on synology nas. For the test I mentioned, I directly run ytdlp without ANY UI (Metube) through command line on my win11 pc use the same setting, which added a name template including 'id' field. The same setting works fine with Metube before.

Are you using the web ui to add the list or posting the list directly to the /add ?

[PikuZheng]

yt-dlp command line has different behavior than embedded yt-dlp.

to make it clear, are you means a playlist works fine with the old version of metube, but the new one doesn't work? can you provide a link for test?

[mxia9416]

Yes, I set up Metube in my NAS like a years ago or so and it has been working without issue. I can't recall the exact version that cause this issue as I set up an auto update of the Metube docker container, I also don't use it daily, maybe every once in 2 months, so I'm also surprised to find it stopped working today. My docker setting are below:

Testing list link will report error below: The individual video in the list downloads fine.

[PikuZheng]

啊 是b站

[mxia9416]

其实我是下P站的时候发现会报错，但感觉影响不好（ 后来发现B站也不行，之前都是没问题的

[PikuZheng]

这个问题根源大概是程序本来设计上是给youtube用的，其他视频网站有的是拿不到id。大体上我能猜到哪个提交改了这个地方。不过简单的方法是打开Strict Playlist mode

[mxia9416]

试了一下，Strict Playlist mode可以把list里面的视频当成单独视频下载。 不过因为我的需求是输入list的url然后批量下载一个list里面的所有视频，这个方法可能并不适用。 我先用yt-dlp本体来下载，期待metube能修复这个问题 。

[PikuZheng]

@labmonkey could you please look into this issue? The 'id' attribute seems to be missing here https://github.com/alexta69/metube/blob/c4561dc9b33bec365ffeabdabc21497d02538df9/app/ytdl.py#L289

[PikuZheng]

试了一下，Strict Playlist mode可以把list里面的视频当成单独视频下载。 不过因为我的需求是输入list的url然后批量下载一个list里面的所有视频，这个方法可能并不适用。 我先用yt-dlp本体来下载，期待metube能修复这个问题 。

它会一个个全下载下来的，但是下载列表中只有一行。 或者你回退到这个版本试试 ghcr.io/alexta69/metube:2024-08-07

[mxia9416]

metube:2024-08-07

Version metube:2024-08-07 has no issue and acts like before. I'll keep on this version and pause auto-update until it's solved. Thanks!

[denniseffing]

The issue is caused by the following line: https://github.com/alexta69/metube/blob/d0a2de2acdfc71af4b144e16ee154b75dd56a1ef/app/ytdl.py#L286-L286

Some extractors do not return an id property when returning an entry of type url (e.g. the Patreon extractor). This worked fine before this line was added in 301ff92b.

I am not sure what the correct fix is. According to the code comment, removing this line would break the OUTPUT_TEMPLATE_PLAYLIST functionality because all playlist_ metadata appended afterwards would be lost. However, I think that the line is a very dirty hack anyways and it does not surprise me in the slightest that it breaks things. And breaking the OUTPUT_TEMPLATE_PLAYLIST functionality is still better than breaking adding playlists altogether. @alexta69 what are your thoughts on this?

[PikuZheng]

Some extractors do not return an id property when returning an entry of type url (e.g. the Patreon extractor). This worked fine before this line was added in 301ff92.

I think @labmonkey 's commit cause this issue

I am not sure what the correct fix is.

me too... I can't follow his train of thought. This issue only occurs in non-youtube (i.e. no video id) situations. I think setting a default value might help?

[PikuZheng]

also strict mode failed after this commit

[Cinj216]

So is anything ever going to be done about this issue or is the creator going to stick to his guns of not reverting/fixing bad code?

[mxia9416]

I made a test container with the latest image (the one that added concurrent download mode) and deleted the line mentioned below. It worked for me to bypass the error of missing 'id' in my use case. I don't know if there will be other issue caused by deleting this line, but if anyone on the same boat wants a temperary fix, this could be worth trying.

metube/app/ytdl.py

Line 286 in d0a2de2

etr["_type"] = "video" # Prevents video to be treated as url and lose below properties during processing