Unwatched videos being marked as watched anyway

[Parent5446]

Overview

I can't quite tell what part is broken here, but I'm experiencing a fairly common situation where videos on the YouTube homepage are being marked as watched by the extension, despite not having ever watched them.

As an example, here is a video being shown as watched:

But when I search it in the extension's database, it doesn't show up:

And just as a sanity check, the video isn't in my Firefox browser history either (and I only watch YouTube in the browser):

I thought perhaps the database search feature in the extension's page might be broken, but if I search for videos that I have definitely watched, they do show up. So I can only assume there's some problem with the matching method in which the extension is checking that a video exists in the database.

My own debugging

I started looking at youtube.js and thought maybe it was being picked up by objProgresscache, but when I look in the console, the video in question (https://www.youtube.com/watch?v=GEfYa07fjd4) doesn't show up:

[Parent5446]

A quick followup: this only happens on the YouTube homepage. If I go directly to the user's channel page, or if I find the video in my Subscriptions feed, it doesn't show up as watched.

[sniklaus]

Thank you for providing all this information! The issue isn't that the extension thinks that you watched the video, the issue is that YouTube will reuse HTML elements. In essence, there might have been a video that you actually watched and the extension marked it accordingly. Then, the element becomes no longer necessary (for example because you navigated to a different page) and instead of deleting it, YouTube places into a cache. Then, when a new video is ought to be shown in the list, YouTube first reuses previously used elements from its cache (for the sake of efficiency, probably). However, the HTML element in the cache has already been marked as watched but it is now being used for a different video. This is, unfortunately, quite difficult to detect so my current implementation may not work reliably on YouTube's homepage. I will have to look into this further in the future. :+1:

[Almashina]

Hi there! The same issue appear when use in YT button "back" ("previous page") for return to YT main page, for example. Some "memory effect" when navigating backward around YT in the same tab.

[drejjmit]

This is still a problem on main page. Can't you just check the video-title-link?

[ghost]

I notice that problem only in sidebar to the right when already watching some video. But not in search results. Using option like "hide related videos" in Enhancer for Youtube extension is good temporary solution, but not really a solution.

[jmg999]

I've also experienced this problem in a couple of different ways. First, videos that show up as recommendations in the right-hand column are beginning to be marked as watched, and second, I'd noticed that if I pulled up a playlist and played a single video, all the videos in the playlist would be marked as watched. This became really confusing to try to organize, b/c some playlists have dozens of videos.

[JustAnOkapi]

I get videos I have never seen that came out hours ago marked as watched. They are normally at the top of the sidebar recommendations.

[JustAnOkapi]

A few minutes later when I exited full screen the video was no longer marked.

[sniklaus]

Please see: https://github.com/sniklaus/youtube-watchmarker/issues/56#issuecomment-770324096

[JustAnOkapi]

This is not on the homepage. Its on the recommendation sidebar.

[sniklaus]

The explanation in the linked comment also applies to the sidebar.

[qstar-inc]

I've seeing this issue on the subscription page as well; I understand how it happens on sidebar/homepage; but why subscription? Also, when the position of that particular video moved (when other subscribed channels added a new video), it no longer have the Watched fade.

[sniklaus]

I just released version 4.3.1 based on some great contributions from @lbmaian that may have addressed this now. Could you give the new version a try and report back? Thanks!

[lbmaian]

@sniklaus FWIW I'm not sure if my changes would've fixed this. I personally never use the YT homepage so I don't recall encountering this issue.

Perhaps one way to ensure this is fixed is to monitor for yt-navigate-finish events (window or document.addEventListener('yt-navigate-finish', ...)) and clear boolMarkcache there.

[JustAnOkapi]

the issue seemed to be caused by yt caching thumbnails

[lbmaian]

Oh I didn't see later comments that this happens for video recommendations and subscriptions. I've only seen videos NOT being marked as watched despite being recorded as such in watchmarker. That should've been due to thumbnail image caching (by the browser, not YT), which should've been fixed.

What's likely happening in this issue is outlined in https://github.com/sniklaus/youtube-watchmarker/issues/56#issuecomment-770324096 - HTML elements being reused when "internally" navigating between YT pages. Hence my suggestion of clearing the cache whenever the page changes via a yt-navigate-finish event listener.

[sniklaus]

Perhaps one way to ensure this is fixed is to monitor for yt-navigate-finish events (window or document.addEventListener('yt-navigate-finish', ...)) and clear boolMarkcache there.

Thank you for the suggestion, that is a nifty idea! I am currently using the following mechanism to detect changes which is definitely less efficient but maybe more general. I'll also add the yt-navigate-finish listener. :+1:

https://github.com/sniklaus/youtube-watchmarker/blob/0cb11b1f30c17243505710aa14741a3ac9c5c2b8/youtube.js#L89-L104

[lbmaian]

Well yt-navigate-finish doesn't catch everything, since more videos are loaded as you scroll down, so you still do need the existing interval check. There's might be other custom events that can catch when the new videos are actually inserted into the page, or a MutationObserver to monitor for them, but AFAIK the custom yt-navigate-finish event should catch all the cases where videos are wiped or replaced on the page.

[lbmaian]

Actually I take that back, changing sort order probably also reuses HTML elements. I believe the yt-service-request-completed custom event can catch such cases, firing after the HTML elements are added/replaced (I think? needs to be checked), though the event doesn't always fire when scrolling down to load more videos (YT page internally may include more video data than it actually creates video HTML elements for, only inserting them when scrolling down).

So maybe a combination of existing check and new yt-navigate-finish + yt-service-request-completed listeners would suffice?

edit: also yt-navigate-finish handles the YT tab switching case (it does change the url) edit2: yt-playlist-data-updated also works in lieu of yt-navigate-finish and it fires a bit sooner than the latter

[utkarshgupta137]

I have started seeing this on my subscriptions page too recently. I also use a few other YouTube extensions, most prominently PocketTube to hide the watched videos completely. The thing is that the watched marker comes on after a few minutes (I have a pinned tab with the sub-page always open). I'll check if there is any pattern or anything.

[sniklaus]

I will incorporate @lbmaian's excellent suggestions and work on a new version tonight. It will invalidate the cache when receiving either the 'yt-service-request-completed' or the 'yt-navigate-finish' event. Please let me know whether the new version, which may take a few days to land, helps with this issue once and for all. Thanks!

[Terrance]

Just got the 4.4.1 update and that seems to handle switching between channel video tabs nicely. 🙂

[sniklaus]

Thanks for letting me know! I will close this issue for now but am sure we'll have to revisit this in the future again. So please don't hesitate to comment if the issue persists and I'll reopen it. :+1: