mediacloud
treat YT videos encountered when spidering as special content
When we find a YouTube video in a topic, we should parse out the channel name and create that as a YT source that you can aggregate by.
just noting here that once we assigning youtube channels to media sources, we could actually create a meaningful first class youtube topic by creating a 'url sharing' topic that just treats each youtube post as a link to itself.
-hal
On Tue, Oct 1, 2019 at 8:58 AM rahulbot [email protected] wrote:
Assigned #616 https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_berkmancenter_mediacloud_issues_616&d=DwMCaQ&c=WO-RGvefibhHBZq3fL85hQ&r=0c5FW2CrwCh84ocLICzUHjcwKK-QMUDy4RRw_n18mMo&m=JyagQx0GHC7bsgcmdPxkmWY3ME055zThjxkcHDASvpA&s=r47w2px64XhipbU0N5dOJ8GgJG8Ma1jsocpCHI2UAXg&e= to @hroberts https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_hroberts&d=DwMCaQ&c=WO-RGvefibhHBZq3fL85hQ&r=0c5FW2CrwCh84ocLICzUHjcwKK-QMUDy4RRw_n18mMo&m=JyagQx0GHC7bsgcmdPxkmWY3ME055zThjxkcHDASvpA&s=fQPCBtq9dSWWGiOZptjiRL7uLExIsliX6cThHRDKRrw&e= .
— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_berkmancenter_mediacloud_issues_616-3Femail-5Fsource-3Dnotifications-26email-5Ftoken-3DAAN66T3EHROGJ3KPP6LC3PLQMNJQ3A5CNFSM4I4J475KYY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOT6JASIY-23event-2D2677147939&d=DwMCaQ&c=WO-RGvefibhHBZq3fL85hQ&r=0c5FW2CrwCh84ocLICzUHjcwKK-QMUDy4RRw_n18mMo&m=JyagQx0GHC7bsgcmdPxkmWY3ME055zThjxkcHDASvpA&s=_aNgr1HuVJQfRusN_0BMWFBNL8YDI88B5xmtSA8PG18&e=, or mute the thread https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AAN66T7AB3NV53H7VMTL5XLQMNJQ3ANCNFSM4I4J475A&d=DwMCaQ&c=WO-RGvefibhHBZq3fL85hQ&r=0c5FW2CrwCh84ocLICzUHjcwKK-QMUDy4RRw_n18mMo&m=JyagQx0GHC7bsgcmdPxkmWY3ME055zThjxkcHDASvpA&s=eFouRv1ZSHb_7S9MuSeOI1K4xIBEcAVFKXioQM4FDcs&e= .
Summary: When we see a YT video url in spidering. we should do the following:
- hit the YT API to pull down real metadata
- create a new media source for the channel (perhaps in a new "YouTube Channels" collection)
- assign the story to that channel
Optionally we might want to back port all the existing videos under the monolithic YouTube source we have right now to their own channel media sources.
This is related to #618 (scraping all videos in channel).