xbvr
xbvr copied to clipboard
xbapps
Multiple Scrapers not working in 0.3.19
Hi. Downloaded the software and tried to match up my library of 300 files. I scrapped nearly all the scrappers and found problems with the following scrapers. Each of these scrapers returns no results. Checking the logs as it runs looks like its visiting the sites, but nothing is downloaded at all.
DDFNetworkVR SexBabesVR Virtual Taboo VRConk VRHush VRP Films
Furthermore, I found the following scenes below to be missing, despite the scrapper generally working and downloading other scenes just fine from these sites.
SLROriginals - Asian Delight Royal Flush StripzVR - Open wide StripzVR - Sex kitten VRBangers -Milfy Way VRLatina - Elf On Myself VRLatina - Slippery Sunday
I just tried the ones you listed on a fresh database and SexBabesVR and VRHush are the only ones that don't work for me, all the others work fine.
I'm using docker if that matters. Here is the log of all I mentioned. 0 scenes pulled between them all. Other scrapers working and pulling. This was done on a new container with a fresh pull just now.
time="2022-01-17T00:13:24-08:00" level=info msg="Scraping started at Mon Jan 17 00:13:24 2022" task=scrape time="2022-01-17T00:13:24-08:00" level=info msg="Starting SexBabesVR scraper" completed=false progress=0 scraperID=sexbabesvr started=true task=scraperProgress time="2022-01-17T00:13:24-08:00" level=info msg="Starting VRHush scraper" completed=false progress=0 scraperID=vrhush started=true task=scraperProgress time="2022-01-17T00:13:24-08:00" level=info msg="visiting https://sexbabesvr.com/virtualreality/list" time="2022-01-17T00:13:24-08:00" level=info msg="visiting https://vrhush.com/scenes" time="2022-01-17T00:13:24-08:00" level=info msg="Starting VRCONK scraper" completed=false progress=0 scraperID=vrconk started=true task=scraperProgress time="2022-01-17T00:13:24-08:00" level=info msg="Starting VirtualTaboo scraper" completed=false progress=0 scraperID=virtualtaboo started=true task=scraperProgress time="2022-01-17T00:13:24-08:00" level=info msg="visiting https://virtualtaboo.com/videos" time="2022-01-17T00:13:24-08:00" level=info msg="Starting VRP Films scraper" completed=false progress=0 scraperID=vrpfilms started=true task=scraperProgress time="2022-01-17T00:13:24-08:00" level=info msg="visiting https://vrpfilms.com/vrp-movies" time="2022-01-17T00:13:24-08:00" level=info msg="Starting DDFNetworkVR scraper" completed=false progress=0 scraperID=ddfnetworkvr started=true task=scraperProgress time="2022-01-17T00:13:24-08:00" level=info msg="visiting https://ddfnetworkvr.com/" time="2022-01-17T00:13:24-08:00" level=info msg="visiting https://vrconk.com/videos" time="2022-01-17T00:13:24-08:00" level=info msg="Finished VRHush scraper" completed=true progress=0 scraperID=vrhush started=false task=scraperProgress time="2022-01-17T00:13:24-08:00" level=info msg="Finished DDFNetworkVR scraper" completed=true progress=0 scraperID=ddfnetworkvr started=false task=scraperProgress time="2022-01-17T00:13:24-08:00" level=info msg="Finished SexBabesVR scraper" completed=true progress=0 scraperID=sexbabesvr started=false task=scraperProgress time="2022-01-17T00:13:24-08:00" level=info msg="Finished VRP Films scraper" completed=true progress=0 scraperID=vrpfilms started=false task=scraperProgress time="2022-01-17T00:13:24-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=1" time="2022-01-17T00:13:25-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=2" time="2022-01-17T00:13:25-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=3" time="2022-01-17T00:13:25-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=4" time="2022-01-17T00:13:25-08:00" level=info msg="visiting https://vrconk.com/sex-with-slavic-chick-1" time="2022-01-17T00:13:25-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=5" time="2022-01-17T00:13:26-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=6" time="2022-01-17T00:13:26-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=7" time="2022-01-17T00:13:26-08:00" level=info msg="visiting https://vrconk.com/only-for-your-eyes-2" time="2022-01-17T00:13:27-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=8" time="2022-01-17T00:13:27-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=9" time="2022-01-17T00:13:27-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=10" time="2022-01-17T00:13:27-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=11" time="2022-01-17T00:13:27-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=12" time="2022-01-17T00:13:28-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=13" time="2022-01-17T00:13:28-08:00" level=info msg="visiting https://vrconk.com/looking-for-your-cock-3" time="2022-01-17T00:13:28-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=14" time="2022-01-17T00:13:28-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=15" time="2022-01-17T00:13:28-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=16" time="2022-01-17T00:13:29-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=17" time="2022-01-17T00:13:29-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=18" time="2022-01-17T00:13:29-08:00" level=info msg="visiting https://vrconk.com/finger-warm-up-4" time="2022-01-17T00:13:29-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=19" time="2022-01-17T00:13:29-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=20" time="2022-01-17T00:13:29-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=21" time="2022-01-17T00:13:30-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=22" time="2022-01-17T00:13:30-08:00" level=info msg="visiting https://vrconk.com/fun-with-sex-toy-5" time="2022-01-17T00:13:30-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=23" time="2022-01-17T00:13:30-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=24" time="2022-01-17T00:13:30-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=25" time="2022-01-17T00:13:31-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=26" time="2022-01-17T00:13:31-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=27" time="2022-01-17T00:13:31-08:00" level=info msg="visiting https://vrconk.com/may-i-suck-it-6" time="2022-01-17T00:13:31-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=28" time="2022-01-17T00:13:31-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=29" time="2022-01-17T00:13:32-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=30" time="2022-01-17T00:13:32-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=31" time="2022-01-17T00:13:32-08:00" level=info msg="visiting https://vrconk.com/my-pleasure-in-your-hands-7" time="2022-01-17T00:13:32-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=32" time="2022-01-17T00:13:32-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=33" time="2022-01-17T00:13:32-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=34" time="2022-01-17T00:13:33-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=35" time="2022-01-17T00:13:33-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=36" time="2022-01-17T00:13:33-08:00" level=info msg="visiting https://vrconk.com/take-me-baby-8" time="2022-01-17T00:13:33-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=37" time="2022-01-17T00:13:33-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=38" time="2022-01-17T00:13:34-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=39" time="2022-01-17T00:13:34-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=40" time="2022-01-17T00:13:34-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=41" time="2022-01-17T00:13:34-08:00" level=info msg="visiting https://vrconk.com/breakfast-on-the-table-9" time="2022-01-17T00:13:34-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=42" time="2022-01-17T00:13:35-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=43" time="2022-01-17T00:13:35-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=44" time="2022-01-17T00:13:35-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=45" time="2022-01-17T00:13:35-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=46" time="2022-01-17T00:13:35-08:00" level=info msg="visiting https://vrconk.com/united-boobs-of-desire-10" time="2022-01-17T00:13:35-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=47" time="2022-01-17T00:13:36-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=48" time="2022-01-17T00:13:36-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=49" time="2022-01-17T00:13:36-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=50" time="2022-01-17T00:13:36-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=51" time="2022-01-17T00:13:36-08:00" level=info msg="visiting https://vrconk.com/i-change-my-lingerie-three-times-for-you-15" time="2022-01-17T00:13:37-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=52" time="2022-01-17T00:13:37-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=53" time="2022-01-17T00:13:37-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=54" time="2022-01-17T00:13:37-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=55" time="2022-01-17T00:13:37-08:00" level=info msg="visiting https://vrconk.com/take-care-of-the-bunny-11" time="2022-01-17T00:13:37-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=56" time="2022-01-17T00:13:38-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=57" time="2022-01-17T00:13:38-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=58" time="2022-01-17T00:13:38-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=59" time="2022-01-17T00:13:38-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=60" time="2022-01-17T00:13:39-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=61" time="2022-01-17T00:13:39-08:00" level=info msg="visiting https://vrconk.com/pussy-wide-open-12" time="2022-01-17T00:13:39-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=62" time="2022-01-17T00:13:39-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=63" time="2022-01-17T00:13:39-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=64" time="2022-01-17T00:13:39-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=65" time="2022-01-17T00:13:40-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=66" time="2022-01-17T00:13:40-08:00" level=info msg="visiting https://vrconk.com/want-to-know-whats-for-dinner-13" time="2022-01-17T00:13:40-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=67" time="2022-01-17T00:13:40-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=68" time="2022-01-17T00:13:40-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=69" time="2022-01-17T00:13:41-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=70" time="2022-01-17T00:13:41-08:00" level=info msg="visiting https://vrconk.com/your-eastern-maid-14" time="2022-01-17T00:13:41-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=71" time="2022-01-17T00:13:41-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=72" time="2022-01-17T00:13:41-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=73" time="2022-01-17T00:13:41-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=74" time="2022-01-17T00:13:42-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=75" time="2022-01-17T00:13:42-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=76" time="2022-01-17T00:13:42-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=77" time="2022-01-17T00:13:42-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=78" time="2022-01-17T00:13:42-08:00" level=info msg="visiting https://vrconk.com/fun-with-real-vr-amateur-19" time="2022-01-17T00:13:43-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=79" time="2022-01-17T00:13:43-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=80" time="2022-01-17T00:13:43-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=81" time="2022-01-17T00:13:43-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=82" time="2022-01-17T00:13:43-08:00" level=info msg="visiting https://vrconk.com/juicy-holes-22" time="2022-01-17T00:13:43-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=83" time="2022-01-17T00:13:44-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=84" time="2022-01-17T00:13:44-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=85" time="2022-01-17T00:13:44-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=86" time="2022-01-17T00:13:44-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=87" time="2022-01-17T00:13:44-08:00" level=info msg="visiting https://vrconk.com/rabbit-fuck-23" time="2022-01-17T00:13:45-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=88" time="2022-01-17T00:13:45-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=89" time="2022-01-17T00:13:45-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=90" time="2022-01-17T00:13:45-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=91" time="2022-01-17T00:13:45-08:00" level=info msg="visiting https://vrconk.com/amateur-chick-in-the-kitchen-25" time="2022-01-17T00:13:45-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=92" time="2022-01-17T00:13:46-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=93" time="2022-01-17T00:13:46-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=94" time="2022-01-17T00:13:46-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=95" time="2022-01-17T00:13:46-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=96" time="2022-01-17T00:13:47-08:00" level=info msg="Finished VRCONK scraper" completed=true progress=0 scraperID=vrconk started=false task=scraperProgress time="2022-01-17T00:13:47-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=97" time="2022-01-17T00:13:47-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=98" time="2022-01-17T00:13:47-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=99" time="2022-01-17T00:13:47-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=100" time="2022-01-17T00:13:48-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=101" time="2022-01-17T00:13:48-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=102" time="2022-01-17T00:13:48-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=103" time="2022-01-17T00:13:48-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=104" time="2022-01-17T00:13:48-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=105" time="2022-01-17T00:13:49-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=106" time="2022-01-17T00:13:49-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=107" time="2022-01-17T00:13:49-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=108" time="2022-01-17T00:13:49-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=109" time="2022-01-17T00:13:50-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=110" time="2022-01-17T00:13:50-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=111" time="2022-01-17T00:13:50-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=112" time="2022-01-17T00:13:50-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=113" time="2022-01-17T00:13:50-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=114" time="2022-01-17T00:13:51-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=115" time="2022-01-17T00:13:51-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=116" time="2022-01-17T00:13:51-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=117" time="2022-01-17T00:13:51-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=118" time="2022-01-17T00:13:52-08:00" level=info msg="visiting https://virtualtaboo.com/videos?page=119" time="2022-01-17T00:13:52-08:00" level=info msg="Finished VirtualTaboo scraper" completed=true progress=0 scraperID=virtualtaboo started=false task=scraperProgress time="2022-01-17T00:13:52-08:00" level=info msg=DONE task=scraperProgress time="2022-01-17T00:13:52-08:00" level=info msg="Updating tag counts" task=scrape time="2022-01-17T00:13:52-08:00" level=info msg="Building search index..." task=scrape time="2022-01-17T00:13:52-08:00" level=info msg="Search index built!" task=scrape time="2022-01-17T00:13:52-08:00" level=info msg="Reapplying edits" task=scrape time="2022-01-17T00:13:52-08:00" level=info msg="Scraped 0 new scenes in 28s" task=scrape time="2022-01-17T00:14:33-08:00" level=info msg="Lost 6115821042194963" time="2022-01-17T00:18:13-08:00" level=info msg="Start scanning volumes" task=rescan time="2022-01-17T00:18:13-08:00" level=info msg="Matching Scenes to known filenames" task=rescan time="2022-01-17T00:18:13-08:00" level=info msg="Update status of Scenes" task=rescan time="2022-01-17T00:18:13-08:00" level=info msg="Generating heatmaps" task=rescan time="2022-01-17T00:18:13-08:00" level=info msg="Scanning complete" task=rescan time="2022-01-17T00:18:13-08:00" level=info msg="Saved state" time="2022-01-17T00:18:13-08:00" level=info msg="Start scanning volumes" task=rescan time="2022-01-17T00:18:13-08:00" level=info msg="Matching Scenes to known filenames" task=rescan time="2022-01-17T00:18:13-08:00" level=info msg="Update status of Scenes" task=rescan time="2022-01-17T00:18:13-08:00" level=info msg="Generating heatmaps" task=rescan time="2022-01-17T00:18:13-08:00" level=info msg="Scanning complete" task=rescan time="2022-01-17T00:18:13-08:00" level=info msg="Saved state"
What docker image are you using?
If VirtualTaboo is not scraping, you probably aren't using 0.3.19. Try updating again. We are at 0.3.20 now.
I scrapped nearly all the scrappers and found problems with the following scrapers.
The only scraper that is "broken" is SexBabesVR. It is included in the next release (0.3.21).
VRHush/VRAllure has implemented Stackpath which breaks scraping.
Furthermore, I found the following scenes below to be missing, despite the scrapper generally working and downloading other scenes just fine from these sites.
Scenes missing from SLR (possibly due to the current removal of A LOT of videos for compliance): SLROriginals - Asian Delight Royal Flush StripzVR - Open wide StripzVR - Sex kitten
Missing from VRBangers website - Milfy Way Missing from VRLatina website - Elf On Myself & Slippery Sunday
I just started a fresh docker running the latest 0.3.23. All the scrapers mentioned earlier (aside from VR hush) appear to be working now, thanks
Can VRHush be removed from the scraper list if its non functioning?
Also, is there a way to scrape the missing scenes manually by entering a page url? Other sites like VRporn.com have these missing scenes listed with pics. Even if I could just add the data for a scene manually by uploading screenshots and then entering the description, actor names and tags
Otherwise I guess this issue can be closed and those things made as separate requests.
To clarify, do I need a new database that the scrappers work again?
The following scrappers don't work on my setup and XBVR 0.3.23
VRHush, VRAllure StripzVR (can find some scenes, but most scenes are not found)
I reset the search index, without changes. Are there any solution for StripzVR?
you sure can.
https://metadataapi.net/ -- click login on the top right (create account) As soon as you're logged in (email addy has to be verified) click on your profile icon ( top right ) Click "API Tokens" and create one, copy the string over to N++ and in a new tab open XBVR
In XBVR WebUI click options the create/import scene paste your API token in the field for TPDB API and in the box below that a link to the scene.
Every site that I've tried the scene was scrapped successfully -- not just from ThePornDB Only need to have the API Token from TPTB and (so far) XBvR has saved the token
