scrapyrt

scrapyrt copied to clipboard

can scrapyrt be used to schedule scrapes?

1

I am using scrapyrt extensively by just sending requests from another server where that other server has been using crontab. However, I began wondering if I could just set scrapyrt...

doverradio

def modify_realtime_request(self, request): yield {'name':'test'} {"status": "ok", "items": [], "items_dropped": [], "stats": {"elapsed_time_seconds": 0.004274, "finish_reason": "finished", "finish_time": "2022-10-27 08:50:15", "log_count/ERROR": 2, "log_count/INFO": 9, "memusage/max": 65720320, "memusage/startup": 65716224, "start_time": "2022-10-27 08:50:15"},...

internalG

fix for #162 Need to add unit test for it

pawelmhm

Unable to override settings file

2

Hi Team, I am trying to override the log settings so that I can dump my logs to a custom file in a custom directory. However, the scrapyRT code never...

gleake-reorg