Hope that users can be given the option to more precisely control crawling behavior
- [ ] It is hoped that when crawling works, information such as the title of the work, the name of the author, and the avatar can be obtained at the same time.
- [x] Some users using an external network environment may be given the option to control whether to enable the proxy.
Thanks for opening this issue and sorry for the delayed reply.
You are welcome to drop a PR if you would like to help enhance it. Besides, it would be better to briefly introduce your PR design here before hands-on it.
As for these two issues, existing APIs can partially resolve them.
-
By enabling
download_config.with_tag, tags of artworks, including avatar description, will be collected. However, it seems the title and author information are missing. I believe expandingwith_tagoption intowith_metadataoption could be a elegant solution. -
Proxy can be disabled by setting
network_config.proxy["https"] = "", but this still lacks some flexibility for more complicated scenarios.
need with_metadata for dataAnalysis
need
with_metadatafor dataAnalysis
What kind of metadata do you need? 👀