crawler icon indicating copy to clipboard operation
crawler copied to clipboard

`init_hook` method called during class instantiation

Open oiwn opened this issue 5 years ago • 0 comments

https://github.com/lorien/crawler/blob/master/crawler/base.py#L86

init_hook method called inside __init__

So for instance if you're doing some work (db calls etc.) inside init_hook there is no way to change class attributes before calling init hook so the following code will not work.

@cmd.command()
@click.option('--num', type=int, default=1000,
              help='Batch size')
def profiles(num):
    from crawlers import YourCrawler

    bot = YourCrawler(
        num_network_threads=10)
    bot.number_per_page = num

    bot.run()

oiwn avatar Oct 13 '18 02:10 oiwn