Feature request: Sort subscriptions into categories

[trwnh]

Rationale

It would be nice to be able to sort one's own subscriptions into categories. One problem I personally have is that I subscribe to a wide variety of content that publishes at different rates, and what I watch depends on my mood and how much free time I have. With only one subscription feed, longform content is much more likely to remain unwatched because I don't happen to have the time for it when I see it, and then shorter content pushes it down beyond view and I forget about it when I have time to watch. The solution I've used on YouTube is to add these videos to the Watch Later list, but then my Watch Later list starts getting too big, so I make a Watch Later (Long) list, and then when I find myself in a certain mood I have to look through the two lists, so I make various Watch Later lists based around category, and the whole thing becomes a mess.

Proposal

Allow users to create different categories or subscription feeds, then sort their subscriptions into these various feeds.

Example:

• A feed for Gaming
• A feed for Math & Science
• A feed for Technology
• A feed for Music
• A feed for Comedy
• A feed for Politics etc, as each user decides, similar to adding feeds to a specific folder in a feed reader.

[elypter]

i think this serves a similar purpose as personal playlists. the most straight forward way to implement this is probably to have invivious playlists which were suggested in other issues before and give them a flag that makes them a subscription target and another flag that makes them the default subscription target. if the use clicks the subscribe button the channel will be added to the default subscription-enabled-playlist and if he clicks on an arrow next to it he will see a drop down menu with all the other subscription-enabled-playlists. the current subscription page could serve as a master collector where you can find everything.

[m0se]

This would be a huge feature for invidious. I have the same problem op describes,..

I even built my own workaround for this problem by running my invidious instance on multiple sub-domains with separate accounts so I can have multiple subscription feeds. After authing each account on it's sub-domain and hacking some links to all sub-domains in to the frontend I was able to just switch between different subscription feeds, which was a completely new watching/browsing experience. But maintaining subscription and several other things are a pain in this setup. So a actual solution would be great!

I'm not sure if playlists is the most straight forward way to implement this, as I don't see playlists being involved in the subscription mechanism (as far as I understand the code).

A relatively simple and universal approach I think could be a "tag" based solution where:

• A single subscription can have tags assigned (none, one or many)
• The subscription feed would then includes a tag filter (via URL-params and/or UI)
• The final step then would be to allow the user to add navigation elements with different tag combinations that point to the accordingly filtered subscription feed.

I'm tempted to try doing a PoC hack of this, but this is the 1st time I come across Crystal so I have no idea if I'm actually able to.

[omarroth]

Internally this feature would probably be implemented in the same way as playlists.

I like idea of using tags. As mentioned you can have tag1+tag2+tag3.

Hopefully I'll be able to look more into this soon. For a PoC:

Move subscriptions into a separate table, something like:

CREATE TABLE subscriptions (
    user_email text not null references users,
    channel_id text references channels,
    tags text[],
    -- ... Any other data, author, etc.
    UNIQUE (user_email, channel_id)
);

Although from a quick test it appears users and channels don't have a primary key set:

ALTER TABLE users ADD PRIMARY KEY (email);
ALTER TABLE channels ADD PRIMARY KEY (id);

See here.

SELECT subscriptions FROM users WHERE email = $1 would become something like SELECT ucid FROM subscriptions WHERE email = $1. Additionally subscription_ajax becomes an INSERT INTO subscriptions ....

For generating feeds you need an extra WHERE clause:

SELECT * FROM subscriptions_139a... WHERE ucid = ANY (SELECT channel_id FROM subscriptions WHERE user_email = $1 AND $2 = ANY(tags));

Likely $2 = ANY(tags) should be '{tag1,tag2}' <@ tags for supporting multiple tags.

Hopefully that's enough to get you started if you'd like to take a look. As mentioned I'm hoping to have time to look more into it but may not be able to for a while. Feel free to ask any questions here or in the Matrix server.

[m0se]

Thanks for the detailed input. I already had a go at a little test implementation last week. My assumption was, that this is not a core feature and therefore should not interfere/alter the existing data model in a way that complicates upgrade paths for existing installation, so I choose more of a append approach.

I did wonder why subscriptions are kept in a text array rather then a separate table, but didn't assume that this feature would justify changing that.

So I manged to get a basic tags based filter working for /feed/subscriptions?tags=tata,mumu with the following approach:

Add a channel_tags table:

CREATE TABLE public.channel_tags
(
  email text NOT NULL,
  ucid text NOT NULL,
  tags text[],
  CONSTRAINT tags_id_key UNIQUE (email,ucid)
);

Then added manually two test records:

 email |           ucid           |        tags        
-------+--------------------------+--------------------
 test  | UCSMOQeBJ2RAnuFungnQOxLg | {lala,blabla,tata}
 test  | UCqrrxZeeFSNCjGmD-33SKMw | {lala,mumu}

To the /feed/subscriptions endpoint I added the following WHERE clause part:

tags = env.params.query["tags"]?
tags ||= ""
if tags == ""
  tags_clause = ""
else
  tags_clause = "AND #{view_name}.ucid = \
      ANY (SELECT ucid FROM channel_tags WHERE email = '#{user.email}' AND tags && '{#{tags}}'::text[])"
end

Which I added to all the queries that fetch videos for the feed (I ignored notifications for now):

videos = PG_DB.query_all("SELECT DISTINCT ON (ucid) * FROM #{view_name} WHERE \
NOT id = ANY (#{values}) \
#{tags_clause} \
ORDER BY ucid, published DESC", as: ChannelVideo)

since the tags_clause is empty when no tags are given, the use without tags is not influenced.

For managing tags (on the channel and subscription manager page) I was thinking about adding get, post and delete endpoints and interact with them via ajax.

I already did a quick implementation of a get "/tags/:ucid" endpoint to fetch all tags of a given channel for the signed in user, but then started wondering if loading stuff via ajax fits the with current frontend concept, as from what I can see ajax stuff is kept to a minimum. Howerver using ajax for adding and deleting tags is probably the proper approach.

So my main questions would be:

1. Is a reorganization of the data model pending anyway and subscriptions will be moved to a seperate table or should I continue with the non-disruptive approach?
2. Is it ok to load tags for display with ajax or would having tags rendered in to the page server-side be preferred?

[omarroth]

Apologies for the delay.

Reorganization is pending, since it makes any future features regarding subscriptions much easier to add. Even if it isn't justifiable for just this feature I think it's worth doing.

Where possible it should be possible to use a feature without JS, so I would prefer tags be rendered server-side if possible. Ajax on top of that is fine though, and using what you've already implemented sounds good for adding and deleting tags.

You may want to look into extending /subscription_ajax with two actions for adding and removing tags (action_create_tag and action_remove_tag or similar), since it already has handling for CSRF and supports use through a <form>, which doesn't require JS. See here and here.

[m0se]

Ok, I now spent a few hours looking into the reorg and it seems not that trivial to do. There is a bunch of obvious queries that can just be adapted, but there is also the db_mapping stuff which when changed would impact the rest of the code base.

My first approach was to keep the User struct as is, to avoid impacting other parts of the code base. So I tried adopting the queries which are fetching the records for the User struct:

user = PG_DB.query_one("SELECT updated, notifications, \
                ARRAY(SELECT channel_id FROM subscriptions WHERE email=users.email) AS subscriptions, \
                email, preferences, password, token, watched \
                FROM users WHERE email = $1", email, as: User)

My hope was to keep the db change transparent to the object model, but after stumbling upon the helper-function analyze_table which seems to do some dynamic harmonization between tables and structs I'm starting to wonder if this is the appropriate approach or if a new (nested?) subscription struct needs to be introduced.

It is absolutely possible that missing something here (this is the 1st thing crystal I look at) and there may be a low impact way to do this reorg after all, but from the understanding I gained so far it looks to me like the reorg goes deeper then expected and probably should be treated as a separate thing.

As I think it also needs some clever migration scripts and some sort of regression testing beyond what's needed for the Tag Feature

I'm tempted to go back to the low-impact Tag feature approach, since adapting the low impact tag-feature to a new db-structure in the future would in my estimate be pretty simple. (basically just migrate tags-arrays from channel_tags table to the new subscriptions table and adopt a bunch of queries)

[omarroth]

My recommendation would also be to use the low-impact approach with a schema that's easy to migrate as you suggested. If you'd like to open a PR with only the necessary changes to properly test this feature. I'll plan on doing the reorg separately and we can rebase on top of that.

[m0se]

Great. As far as I can tell at the moment, it's probably gonna be just one text[] field that would start in it's own channel_tags table with PK email+ucid and later on can simply be migrated to the upcoming subscriptions table. There might be more additions needed to the schema for the UI/Navigation part of the Tags feature, but conceptually I haven't thought that thru yet, as I first want to get the add, remove and feed-filter part working.

[naodesu]

Tags or categories would be a killer feature for me. I have too many subscriptions on YouTube and it's painful to manage them.

[ffytczuthp]

More would like to add that it would be useful to put tags/labels/categories not only for subscriptions, but also for individual videos.

Thanks. Very forward to this feature.

[dirtyid]

My goto solution has been videodeck for youtube which has worked flawlessly for years until recently being throttled by Youtube API. There's also pockettube subscription manager extention for youtube itself, but it's much more clumbsy. Hope this feature gets replicated.

[m0se]

Ok, it has been a while, I somewhat forgot about this endeavor, but last Friday I started again based on current master to implement this feature.

I now have a working implementation of the following:

• Adding the channel_tags table on initialization via Docker entrypoint
  • non-docker creation and migration-db-xxx.sh still needs to be done
• A tag-widget similar to the subscribe-widget to add/remove tags on the channel-page (with and without js)
• A tag-filter option for the /feed/subscription endpoint to get subscriptions filtered by tags
  • Example: /feed/subscriptions?tags=bbg+!wewe+lala+!tata = ( bbg OR lala) AND NOT (tata OR wewe)

Now whats missing to have this as a usable feature is some sort of integration into the navigation.

My personal use-case for a tag-based navigation is to be able to quickly "zap" threw different subscription-feeds. For more flexibility I was thinking about a two level approach, like this:

tag_nav_1:tag1+tag2+!tag3
tag_nav_2:tag3+tag2+!tag1
tag_nav_3:tag4+!tag5+tag2

The most simple way I can think of to make this manageable by the user would be a json-string, like the example above, in the user preferences. Based on this (if not empty) a drop-down menu could be generated in the navigation area. Additional there could be sub-menu to directly choose single tags.

A alternative (or additional) option would be to user the regular searchbar with a prefix, so tag filters could be typed there like this: tags:tag1+tag2+!tag3 and instead of starting a regular search this would redirect to /feed/subscriptions?tags=tag1+tag2+!tag3.

Now before I start implement one or the other or both I'd like to ask for some input from you @omarroth on how to proceed with this.

• Currently everything I did is local, would it make sense to already cleanup and start a pull-request with the finished components (adding tags + tag filter via url) or should continue locally? (I could also put it in a fork on my github and don't start a pull, if there's a interest looking at the code.)
• Since the ui of the "add/remove tag" part is not very intrusive to the overall layout I did it just like I felt it would fit. (see screenshots, feetback?)
• Dropdown menues on the other hand (if thats a agreeable option in general) would need to go in the main layout, when turned on in user prefs. My prefered placement would be in the top bar (left or right of the searchbar), so they are always accessible from everywhere. But looking at the "feed menu" concept, that would probably be the proper place to have them integrated. So the question is should this be it's own navigation widget, that can be turned on/off or merged with the feed menu?
• I maybe do some inspect-element drafts of how the nav integration could look like and upload screenshots...

[m0se]

So this would be a example of a top-bar tag menu: ..and thats how the configuration of menu entries could look like:

[fabianski7]

hi @m0se,

this still a WIP?

[m0se]

No, sorry, it pretty much stopped at the commit linked above. I was not sure if I should do a pull request back then and never got any feedback.
I run that feature for a while in my setup, but then stopped using invidious all together.

I assume the code base probably changed to much over the last year for this code to still be useful.

[trymeouteh]

I am all for this, and if it could allow you to enable/disable a categories of subscriptions from showing up in your main feed would be even better!

[boniek83]

This is killer feature.

[github-actions[bot]]

This issue has been automatically marked as stale and will be closed in 30 days because it has not had recent activity and is much likely outdated. If you think this issue is still relevant and applicable, you just have to post a comment and it will be unmarked.

[trwnh]

Still relevant

[porchthecoder]

I would love this.

[wbecher]

I would love to see this feature implemented.

[SushiByte-beep]

I need this feature, i am not sure how much i can use invidious without.

Ideally though i would much prefer if the subscription view could be shown in columns of different tags. Much like Pockettube deck (YT extensions)

[digitalindependent]

This would be really lovely.