infinity-scrape
infinity-scrape copied to clipboard
Consider not making "out" unique in db?
Making "out" unique is useful because there is no repetition, however when using the CHECK_IF_ALREADY_DONE flag, some of the previous combinations get lost when a new one is found. If we use the CHECK_IF_ALREADY_DONE flag we want all the different recipes to reach an element in order to reduce the number of requests (which is the search limiting factor).
This would reduce the number of request but would introduce some new problems like which recipie should you choose when using howtoget.py.
For howtoget.py you should choose the recipe with the smallest ID. As recipes are introduced in discovery order, the smallest ID is the oldest known recipe.