icrawler

icrawler copied to clipboard

TypeError: 'NoneType' object is not iterable

23

For me the GoogleImageCrawler of icrawler doesn't work anymore. I updated the user agent in `crawler.py` since that seemed to work in the past, but no luck here. I tried...

sgttwld

I'm currently trying to use icrawler to download images working through a list for each of its keywords. The issue I'm having is that as I only want one picture...

Maxcba

KeyError:'data' when using BaiduImageCrawler

6

``` Traceback (most recent call last): File "/home/minami/anaconda3/envs/python_function/lib/python3.7/threading.py", line 926, in _bootstrap_inner self.run() File "anaconda3/envs/python_function/lib/python3.7/threading.py", line 870, in run self._target(*self._args, **self._kwargs) File "anaconda3/envs/python_function/lib/python3.7/site-packages/icrawler/parser.py", line 104, in worker_exec for task in...

tpnam0901

Excellent !

1

I was trying to get some images for a class project and didn't want to manually download 100 images and your software came to the rescue. Excellent work, nice coding

bytearchive

哈喽！默认爬下来的是国内版必应结果，我想用国际版的结果应该怎么操作呀？非常感谢！！！！！  还有，有的图片是png格式，如何保存透明背景的png图片？感谢！ `from icrawler.builtin import BingImageCrawler ing_crawler = BingImageCrawler(downloader_threads=4, storage={'root_dir': 'your_image_dir'}) bing_crawler.crawl(keyword='clip+art+technology', filters=None, offset=0, max_num=1000) `

sqinghua

search image by date doesnt work, the paramter of Ccd_min and Ccd_max useless, please fix it

MeekoI

Google Crawler can only get around 100 images instead of 1000

7

Hi, when I used the searching URLs generated by `feed()` function in `GoogleFeeder`, I can only get around 100 images although the `max_num=1000`. I find that all the URLs get...

jianjieluo

Crawl only jpg files

2

Hi there! When I use `GoogleImageCrawler`, I get sometimes `png` and sometimes `jpg` files, depending on what google finds. Is there a way to configure the crawler to only download...

LostInDarkMath

Unable to download more images in BingImageDownloader

2

I am trying to create a ML dataset I set max_num to 200 and key='face' it only downloads 109 images , I go to result page where images were being...

sayantanbbb

Download more than 1000 images using BingImageCrawler

4

Hey! I am trying to download 3000 images for each keyword using BingImageCrawler but I am getting cut off at < 1000 images per keyword. The documentation says `To crawl...

animesh-s