How to monitor search results split on multiple pages?

[luboss]

Imagine very common use case - submitting a search form that shows results grouped by 50 for example. GET parameter page=1 shows the first group of 50 results, page=2 shows 51-100 and so on. I'd like to join the full list, sort it and show the difference. Does urlwatch support some kind of "page extender" or list creator based on some specific iterator?

[cfbao]

AFAIK this is the first time someone requested this. I wouldn't say it's a "very common use case".

Some fundamental changes are likely needed to support such use case, so it's unlikely to be implemented any time soon.

[luboss]

Maybe I'm just looking at the wrong place. Even better I might rather try to figure out the API behind the web app that might not be always easy to do. I'd be very surprised if there wasn't any solution. For example watching prices list for products can be very beneficial. If I won't find anything, I'll be happy to start my own project with taking this requirement into account right from the start.

[thp]

You can always write a script (in any language) that does incremental page downloading and merging for that specific page and then use the command line tool output watching option of urlwatch for the monitoring and diffing.

Ľuboš Katrinec [email protected] schrieb am Sa., 13. Juni 2020, 21:07:

Maybe I'm just looking at the wrong place. Even better I might rather try to figure out the API behind the web app that might not be always easy to do. I'd be very surprised if there wasn't any solution. For example watching prices list for products can be very beneficial. If I won't find anything, I'll be happy to start own projects with taking this into account right from the start.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/thp/urlwatch/issues/509#issuecomment-643664922, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABBASPH2O3M7OTP2G6GVNTRWPE6RANCNFSM4N4E6TCQ .