MediaElch icon indicating copy to clipboard operation
MediaElch copied to clipboard

[MovieScraper] Refactor and move to job based scrapers

Open bugwelle opened this issue 4 years ago • 1 comments

Just like for TV scrapers. This will allow us to have parallel scraping, parallel searches and fewer race conditions that would be possible if we move scraping to a background process (like changing the scraper's language in the settings during scraping).

Should fix:

  • #1162

bugwelle avatar Jan 07 '21 14:01 bugwelle

TODO:

  • [x] Rework CustomMovieScraper
    • [x] Let it create multiple scrape jobs
  • [x] Rework CustomMovieScraper handling in MovieDialog -> if a scraper needs search first, let the user select correct movie
  • [x] Save select scraper language when movie is scraped
  • [x] Scrape Dialog: List scrapers that need title search (doesn't match currently; also on first dialog being shown: no item listed; when changing to other scraper, text not removed)
  • [x] Custom Single Movie Scraper
  • [x] Custom Multi Movie Scraper

bugwelle avatar Jan 31 '21 14:01 bugwelle

Ok, so far I think I've implemented everything. Now I need to do some testing:

  • [x] Add scraper tests
  • [x] Manual test
    • [x] Single movie scraping
      • [x] Load all details
        • [x] from the same scraper
        • [x] from different scrapers (custom movie scraper)
      • [x] Load no details (via settings)
    • [x] Multi movie scraping
      • [x] Load all details
        • [x] from the same scraper
        • [x] from different scrapers (custom movie scraper)
      • [x] Load no details (via settings);
        • [x] TODO: Currently still loads items? --> was the case if only one scraper was selected --> fixed

bugwelle avatar Mar 25 '23 15:03 bugwelle