django-dynamic-scraper icon indicating copy to clipboard operation
django-dynamic-scraper copied to clipboard

How to use the scheduling parameters to only run scraper for only 1 time ?

Open vaimdev opened this issue 7 years ago • 1 comments

I have tried to read the Documentation and still cannot understand how can I adjust the parameters to only run the scraper for only 1 time.

I have a scraper just to scrape google search result with fix keyword. I only need to run it for 1 time but I need to schedule it.

I come across this in the documentation : If you don’t need dynamic scheduling, you can also just set the MIN_TIME and the MAX_TIME to the same values and just ignore the rest.

Does it mean if I set MIN_TIME = MAX_TIME , then it will only run 1 time ?

"MIN_TIME": 1440, "MAX_TIME": 10080, "INITIAL_NEXT_ACTION_FACTOR": 1, "ZERO_ACTIONS_FACTOR_CHANGE": 5, "FACTOR_CHANGE_FACTOR": 1.3

vaimdev avatar Jul 15 '17 09:07 vaimdev

If u set MIN=MAX then it means that your spider ll run every a fixed amount of time ex 15 min and it disables dynamic scheduling.

bezkos avatar Jul 16 '17 01:07 bezkos