Joris Rau

Yes. I think that would be a good idea. It may increase the startup time for spiders but it would make frontera easier to use because this issue would no...

Hi. Yes I had lots of other stuff to do :) . ``` scrapy crawl [my spider] -s FRONTERA_SETTINGS=[my project].frontier.spider_settings -s SPIDER_PARTITION_ID=0 ``` ..does not work. It throws: ``` exceptions.TypeError:...

Yes, it throws the same error when I use: ``` scrapy crawl [my spider] -s SPIDER_PARTITION_ID=0 ``` My guess is that there is a difference between scrapy settings (e.g. `SEEDS_SOURCE`,...

With the newest update it now uses the correct `SPIDER_PARTITION_ID` in `messagebus.py`. However it still throws an error (but a different one): ``` ... File "/home/jrisr/Crawl/debug/frontera/frontera/core/manager.py", line 24, in __init__...

Passing the settings via command line works now, but the `settings.py` takes precedence over the command line options, which should not be the case [according to scrapy docs](http://doc.scrapy.org/en/latest/topics/settings.html#populating-the-settings). I would...

Okay. So the `FRONTERA_SETTINGS` have precedence over all the scrapy settings (including the command line settings). In my opinion it would be a good idea to mention that in the...

Yes HBase is the root of the problem. The way I see it, is that HBase fails, which in turn makes the strategy worker fail. The strategy worker should then...

I think handling this error would solve the problem. I would handle it by just exiting.

You are right. Reconnecting to HBase would probably be the better idea here.