v 0.24.0 gives me double videos in feed list (when same channel in multiple channel groups)

[SwallowYourDreams]

Checklist

• [X] I am able to reproduce the bug with the latest version.
• [X] I made sure that there are no existing issues - open or closed - which I could contribute my information to.
• [X] I have read the FAQ and my problem isn't listed.
• [X] I have taken the time to fill in all the required details. I understand that the bug report will be dismissed otherwise.
• [X] This issue contains only one bug.
• [X] I have read and understood the contribution guidelines.

Affected version

0.24.0

Steps to reproduce the bug

1. Updated to v0.24.0
2. Refreshed feed list of subscribed channels

Expected behavior

List of all new videosz one entry per video.

Actual behavior

Some videos appear twice in the list.

Screenshots/Screen recordings

Screenshot here: https://www.reddit.com/r/NewPipe/comments/xnq9l0/why_this_after_updating_to_v24/

Logs

No response

Affected Android/Custom ROM version

CalyxOS 3.9.0

Affected device model

Google Pixel 4a

Additional information

No response

[opusforlife2]

No videos appear twice for me. The first thing to make sure is that your database hasn't been messed around with. You're only supposed to import databases from release versions of Newpipe, not debug versions, as that is unsupported. Please confirm that this is not the case.

If it isn't, this might need a sample database to debug. Let's see what devs say.

[tomsom]

I have the same(similar?) issue that (some) videos appear multiple times in "What's New" but as far as I can tell only videos that have been loaded before updating to 0.24.0 are affected. Also un- and then resubscribing removes the doubles(of that channel). And as of now newly loaded videos don't seem to multiply for me - so in my case it's not a huge issue.

[Stypox]

Could you provide a copy of your database, so that I can try to investigate?

[gigiboeru]

I've just updated to 24 and I'm experiencing the same issue (I didn't import anything, just updated and refreshed the subscription)

LE It looks like it's caused by channel groups, once I deleted all my channel groups, I no longer get double videos.

[SwallowYourDreams]

Could you provide a copy of your database, so that I can try to investigate?

Will do once I get around to it. ETA tonight.

[SwallowYourDreams]

LE It looks like it's caused by channel groups, once I deleted all my channel groups, I no longer get double videos.

You're on to something, mate. It only happens to channel subscriptions that are part of two or more channel groups. Example: you've subscribed to "National Geographic" and you've placed it both in a channel group called "Documentaries" and one called "Nature". Only those will appear twice in the feed list in v0.24.0.

[SwallowYourDreams]

Could you provide a copy of your database, so that I can try to investigate?

Here you go. Thanks in advance for looking into the issue!

NewPipeData-20220928_222122 - double videos.zip

[woj-tek]

Can confirm that it seems to be happening to subscriptions that are in multiple groups. Also happened on it's own after upgrading to 0.24 version.

[solid-snail]

From looking at #8889 the change to a left join looks kinda sus (in the context of this bug). I'd probably start looking there for the source of the bug. Apology for not just testing it and opening a PR. I'm not really setup for android development right now.

https://github.com/TeamNewPipe/NewPipe/pull/8889/files#diff-f85f6bf4cea29ac78e9c829a906aa4e6f076b01c62a8f1a3e53d31ce7e501e1d

in app/src/main/java/org/schabi/newpipe/database/feed/dao/FeedDAO.kt:

-        INNER JOIN feed_group_subscription_join fgs
+        LEFT JOIN feed_group_subscription_join fgs
         ON fgs.subscription_id = f.subscription_id
-        WHERE fgs.group_id = :groupId
+        WHERE (
+            :groupId = ${FeedGroupEntity.GROUP_ALL_ID}
+            OR fgs.group_id = :groupId
+        )

[Stypox]

@solid-snail I just came to the same conclusion separately, but thanks for looking into it! I opened a PR here: #9230

Could you (and everybody else) please test it? Do not just verify whether this bug is fixed, but also that other feed features work properly. Thank you!