webb
webb copied to clipboard
Redundant functions
find_next_link() and get_next_link() are identical functions, one of them can be removed.
Same goes for find_all_links_as_list() and get_all_links().
Also, instead of get_next_link(), get_next_heading(), get_next_paragraph(), etc. being separate functions, it could be a single function, get_next_element() or so, which takes a string ('img', 'p', 'a', ..) as a parameter. This would significantly reduce code size.
Definately planned for it! Currently some functions are used by user-model while others are used by internal crawler functions. So once that is synced, merging of functions will be possible! Thanks!
Marking it as and enhancement.