kemayo
Handle multiple entries in next_link
I wanted to use next_selector on a site with the following setup:
story_part_2.html:
<a href="story_part_1.html">Previous</a>
<a href="story_part_3.html">Next</a>
i.e. no obvious attributes to distinguish the "previous" link from the "next" link.
The actual algorithm to process the next link does include loop detection, so it can disregard the "previous" link if it shows up as a match. However, only next_link[0] was checked. I reimplemented this as a recursive function that loops over all entries in next_link and stops once it finds a new one. (It could keep going, but the flow here did not make sense: having multiple new entries in next_link leads to an ambiguity, as they could be processed depth-first or breadth-first, and in any case, it seemed contrary to the logic of next_selector.)
I sort of worry that the problem domain we're in could legitimately hit python's recursion limit here. The limit should default to 1,000 deep, and I checked and the full download of A Practical Guide To Evil fetches 699 chapters. The Wandering Inn has 757 currently, so it's almost certainly actually going to reach that limit someday. So "web serials with in-the-range-of a thousand chapters" are definitely a possibility... (let's not even talk about translated Chinese xianxia).
For your specific case, could you have made the existing next_selector behavior work with some combination of :last-child or :last-of-type? I know that the markup isn't always conducive to actually getting those to cooperate...
There are a bunch of other links (social media sharing, etc.) on the page. They can be excluded by class, but I don't think last selectors would work to find the "next" link.
I had rewritten this to use a recursive function simply because it was easier to express the desired logic, but I can convert it back to a loop if that's the primary concern.