PixivCrawler icon indicating copy to clipboard operation
PixivCrawler copied to clipboard

Hope that users can be given the option to more precisely control crawling behavior

Open nonhana opened this issue 1 year ago • 4 comments

  • [ ] It is hoped that when crawling works, information such as the title of the work, the name of the author, and the avatar can be obtained at the same time.
  • [x] Some users using an external network environment may be given the option to control whether to enable the proxy.

nonhana avatar Jul 17 '24 06:07 nonhana

Thanks for opening this issue and sorry for the delayed reply.

You are welcome to drop a PR if you would like to help enhance it. Besides, it would be better to briefly introduce your PR design here before hands-on it.

cwher avatar Jul 19 '24 03:07 cwher

As for these two issues, existing APIs can partially resolve them.

  1. By enabling download_config.with_tag, tags of artworks, including avatar description, will be collected. However, it seems the title and author information are missing. I believe expanding with_tag option into with_metadata option could be a elegant solution.

  2. Proxy can be disabled by setting network_config.proxy["https"] = "", but this still lacks some flexibility for more complicated scenarios.

cwher avatar Jul 19 '24 03:07 cwher

need with_metadata for dataAnalysis

FCYXSZY avatar Dec 04 '24 12:12 FCYXSZY

need with_metadata for dataAnalysis

What kind of metadata do you need? 👀

cwher avatar Dec 04 '24 23:12 cwher