xbvr icon indicating copy to clipboard operation
xbvr copied to clipboard

Single Source instead of Individual Scrapers

Open goodtimes2 opened this issue 2 years ago • 2 comments

Why not have a github action scrape all the sources, then update a single file or series of yaml files in the repo? Seems that we are all scraping these sites it might be easier on the websites, and less of a hassle to have it occurred in one place. (Many of the issues on this project are scrape related) Would be interested in discussing this further if that is something this project thinks would be nice to have.

goodtimes2 avatar Feb 28 '22 17:02 goodtimes2

As you suggested Stashdb might be an option longer term. I'm working on my own script to export data from stashdb and generate a json file to import. https://github.com/Tweeticoats/stashdb2xbvr

I've created a sample from vrhush and uploaded this to ipfs for testing

https://cloudflare-ipfs.com/ipfs/bafybeidbncm7h5x3utjjguplpidgbk6bu6svakj45uyugxqrguoytfqove/c85a3d13-c1b9-48d0-986e-3bfceaf0afe5.json

Tweeticoats avatar Mar 04 '22 23:03 Tweeticoats

I very much agree that a central "single source instead of individual scrapers" would make much more sense.

@Tweeticoats : great script which goes in the right direction! But instead of scraping it from sources like StashDB or metadataapi.net - wouldn't it be better (content wise) if the scrape was done from sexlikereal or (my personal preference): vrphub.com ?

gh4ms avatar Apr 13 '22 11:04 gh4ms