Add additional site support

[tcely]

We need to adjust a few things to make the code more flexible for sites other than YouTube.

Rebuilding Twitch support based on the previous pull request (#301) would be a good place to start.

Adding Rumble support (#466) would also be an option.

For better future features, I would like to add these new fields to the Media model:

• metadata_date: when the metadata was retrieved
• formats_date: when the formats in the metadata were last changed

Add to MediaState:

    LIVE = 'live', _('Live (waiting to download)')

Source type:

class Twitch_SourceType(models.TextChoices):
    CHANNEL = 't', _('Twitch channel')

Twitch icon:

'<i class="fab fa-twitch"></i>'

Twitch index URL:

'https://www.twitch.tv/{key}/videos?filter=all&sort=time'

Twitch source display URL:

'https://www.twitch.tv/{key}/videos'

Twitch media display URL:

'https://www.twitch.tv/videos/{key}'

URL validation:

    {
        'scheme': 'https',
        'domains': ('www.twitch.tv',),
        'path_regex': '^\/([^\/]+)(\/videos)?$',
        'path_must_not_match': ('/videos', '/videos/videos'),
        'qs_args': [],
        'extract_key': ('path_regex', 1),
        'example': 'https://www.twitch.tv/twitch/videos'
    },

See also:

• #301
• #433
• #466
• #548

[meeb]

My personal preference would be to remove the manual SourceType maps and just support whatever URLs yt-dlp does upstream. This probably needs checking with yt-dlp that it returns a unique ID per item so the "key" still works.

My original plan was to use the source TLD as the "source key". The interface to this would be a generic box in tubesync that you could paste any URL into which would get passed to yt-dlp quickly to see if it returned some media items, if it did, jump to a page with the first page of results returned and a "does this look like the URL returned the media you want to sync?" yes/no confirmation, then add it as a source if "yes".

This would likely need normalising of the domain using something like the Mozilla TLD list (so m.youtube.com got mapped to youtube.com etc.) then the "source key" would be /some-uri off the domain, with each media item ID be whatever yt-dlp returns.

This should work, in theory, and drop all the strict parsing for channel IDs and what not.

This does open up the possibility of people adding mad URLs that don't really work very well as media sources but it's probably worth it given the hundreds of sites yt-dlp supports. It would also support whatever new sites yt-dlp add support for without tubesync having to add source handlers for them.

[davidstowers879]

Please add this feature! Much appreciated!

[tcely]

This would likely need normalising of the domain using something like the Mozilla TLD list (so m.youtube.com got mapped to youtube.com etc.) then the "source key" would be /some-uri off the domain, with each media item ID be whatever yt-dlp returns.

Your idea of normalization is best handled with the extractor_key yt-dlp returns.