Alexander Sibiryakov

Yeah, the custom partitioner could be introduced here https://github.com/scrapinghub/frontera/blob/master/frontera/contrib/messagebus/kafkabus.py#L187 consider also having a custom fingerprint function and partition based on it https://github.com/scrapinghub/frontera/blob/master/frontera/settings/default_settings.py#L60 the problem of custom partitioning is the need...

can we close it @ZipFile ? How is your project going, BTW?

sure, let's do it!

Seed loaders are Scrapy spider middlewares. All the same rules should apply as to Scrapy middlewares. I need to know your Frontera cluster setup: backends, message bus and run mode...

@wetneb What backend do you use? In case of HBase meta isn't persisted, but in SQLA backend it is. Converters are used in spider processes, and conversion happens all the...

@wetneb oh that's great you found it. https://github.com/scrapinghub/frontera/blob/master/frontera/worker/strategies/__init__.py#L90 It's not transferred for historical reasons, but it makes sense to do so. PR's are always welcome.

Hi Canh, KafkaOverusedBackend is obsolete and going to be removed in the next versions, also could be outdated. Please try MessageBus backend with MESSAGE_BUS set to Kafka. This is a...

@nautilus28 do you have strategy worker running? if so, please check it reads the seeds and generates update_score events in the scoring log.

SW cannot access HBase, that's why. It generates events to scoring log after it fetches states from storage. In a separate issue I mentioned transport & protocol have to fit,...

I have no idea why exactly, but I think this is something connected with usage of old 0.98.x versions. Please check: 1. if tables are created? if not, try changing...