auto-youtube-subscription-playlist-2 icon indicating copy to clipboard operation
auto-youtube-subscription-playlist-2 copied to clipboard

Can't handle videos that are scheduled for later and added to playlist early

Open Fabian42 opened this issue 3 years ago • 21 comments

The latest video in this playlist is private: https://www.youtube.com/playlist?list=PL7VmhWGNRxKi1ikLc2b_pi6EIGOCdF10d

Since that video got added, the script always outputs an error message for it in the debug sheet and adds all other videos, but does not update the timestamp, which is why it then adds those same videos to the playlist again.

Fabian42 avatar Jun 01 '21 14:06 Fabian42

Also, some output about which playlist it is would be nice. I can't use the video ID for anything, because it's a private video. And manually searching through my 83 source playlists and channels is pretty tedious.

Fabian42 avatar Jun 01 '21 14:06 Fabian42

Sorry for the late response, but I just tried the script with that playlist and it works fine. Do you have the updated script? Try running your script with the playlist in https://github.com/Elijas/auto-youtube-subscription-playlist-2/issues/74#issuecomment-767135077 and see if you get the same output as me.

I'll look into adding extra output on source of video but since we add all videos to a gigantic list before adding, it might take a decent amount of changes

victorjzsun avatar Jun 23 '21 02:06 victorjzsun

Yes, that is the output in the current version. But then the timestamp doesn't change and the next time, the script adds all other videos (which are not private) to the target playlist, over and over again.

Fabian42 avatar Jun 23 '21 07:06 Fabian42

The playlist no longer has any private videos, that's why the script now works fine with it. It could probably be reproduced with my example playlist from #74, if the latest update timestamp is set to August 2015.

Fabian42 avatar Jun 23 '21 07:06 Fabian42

Gotcha, had to fix the error count, try it now with the new version.

victorjzsun avatar Jun 23 '21 14:06 victorjzsun

I just got a mail saying that in the last 24 hours, there was 1 error, at "6/27/21 12:52:03 PM UTC". The sheet has both of its last update timestamps set to "2021-06-28T12:26:25+00:00" and the debug sheet shows no errors at all. There were also no new videos added to either of the two target playlists in the last hours. Does it just silently skip the unavailable videos now? Wouldn't that lead to it missing all videos that are scheduled for later, but added to the source playlist earlier?

Fabian42 avatar Jun 28 '21 12:06 Fabian42

Yeah, at least for channel uploads, private videos that are scheduled for later will get picked up by the script after it has been publicly published. Still, you should see a log as shown below in the debug sheet image What makes playlists trickier is that it uses PlaylistItems rather than Videos and so, the items have publishedAt timestamps of when they were added to the playlist, not when the video was published (scheduled or not). Therefore, as you said, it'll miss videos scheduled for later but added to the playlist early. We'd probably need to create some sort of cache to remember the videos that have not been added properly and retry at next update. I'll look into a possible implementation.

victorjzsun avatar Jul 06 '21 17:07 victorjzsun

In that case the previous version was actually better, because there I at least noticed that something was up and could manually do it. Right now it is causing me to miss videos. All I see is "Error: 1 video(s) were not added to playlists correctly, please check Debug sheet. Timestamps for respective rows has not been updated." and then I go to the sheet, but it's hours later and the script has already done more rounds and removed that debug text again.

Fabian42 avatar Jul 06 '21 18:07 Fabian42

I was also thinking that we probably want to persist debug logs, just for my information, about how many times do you run the script every day?

victorjzsun avatar Jul 06 '21 18:07 victorjzsun

Exactly 24 times. Much more would run into rate limits anyway.

Fabian42 avatar Jul 06 '21 18:07 Fabian42

If you have time, could you try using https://docs.google.com/spreadsheets/d/1Idv6VLjQbzkJ5ToGFJui6o-qhPlZaWfBBKqflGt5kUA/edit?usp=sharing. I implemented persistence of debug logs along with a new sheet to view a particular execution's logs. Feel free to do whatever you want to the script, I have another version with the implementation saved. P.S. private videos still fail without error, haven't changed that yet, at least it should show up in debug logs

victorjzsun avatar Jul 06 '21 20:07 victorjzsun

Am I supposed to use that one or make a copy? That sheet has no script triggers and I can't add one without linking it to my account, which gives control over my YouTube account to that sheet.

Fabian42 avatar Jul 06 '21 21:07 Fabian42

Yeah, make a copy, it's probably easier that way

victorjzsun avatar Jul 06 '21 23:07 victorjzsun

The case didn't occur in the last week, but now I have a different issue: The script suddenly tries to go through tons of rows that don't exist and eventually runs into a timeout. Maybe it's because I added another column? I'm at column "BT" now. But even after removing the column again, it still has that issue. Here is the start of the log:

Row: 4
Cannot query for user Row: 5
Cannot query for user Tue Jul 13 16:35:33 UTC 2021
Cannot query for user Row: 5
Acquired 0 videos
Row: 5
Cannot query for user Acquired 0 videos
Cannot query for user Tue Jul 13 16:35:33 UTC 2021
Cannot query for user Cannot query for user Acquired 0 videos
Acquired 0 videos
Row: 6
Cannot query for user No new videos yet.
Cannot query for user Tue Jul 13 16:35:34 UTC 2021
Cannot query for user Acquired 0 videos
Acquired 0 videos
Row: 8

The row numbers are 4, 5, 6, 8, 9, 10, 11, 13, 14, 15, 16, 17, 18, 20, 21, 22, 23, 25, 26, 27, 28, 29, 30, 32, 33, 34, 35, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 49, 50, … After the first group of 3, those are always groups of 4 and 6, alternating. Pretty strange. In reality I have 2 rows.

Fabian42 avatar Jul 13 '21 17:07 Fabian42

It's reading the debugData sheet instead of the correct sheet. Try unhiding the debugData sheet and moving the playlist sheet to the left

victorjzsun avatar Jul 13 '21 17:07 victorjzsun

The order is correct and the only thing hidden are the columns C to E in "Sheet1". The debug sheet is called "Debug", not "debugData" BTW.

Fabian42 avatar Jul 13 '21 18:07 Fabian42

I created a new sheet called debugData to store all of the past debug data (hid it to prevent confusion), the "Debug" sheet is now just a convenient viewer which parses the data in the hidden sheet. You'll be able to see it if you click on the sheet menu button image

victorjzsun avatar Jul 13 '21 18:07 victorjzsun

Now it works again. Weird that that randomly started happening. I'll keep monitoring for what this ticket is actually about.

Fabian42 avatar Jul 13 '21 18:07 Fabian42

No problems with the new debug logger/viewer so far right? other than the sheet reordering

victorjzsun avatar Jul 13 '21 18:07 victorjzsun

I can't see the log of a currently running script in it, so I have to wait to see what did happen instead of seeing it live. Otherwise, only minor stuff. The dropdown list is getting quite long, so having the latest one on top would be nice. And maybe having it selected by default so that the log is already on the right side, that could speed up doing many tests in a row.

Fabian42 avatar Jul 13 '21 20:07 Fabian42

Makes sense, will make some changes

victorjzsun avatar Jul 13 '21 21:07 victorjzsun