site icon indicating copy to clipboard operation
site copied to clipboard

Don't limit feeds to last 5 posts

Open jtagcat opened this issue 2 years ago • 5 comments

It's nice to import a list of everything, to help keep track of what you've gone through.
(used one of my workaround methods for now, essentially crawling)

I looked around for a bit, didn't find the spot where posts / feed items were being limited at.
I don't recall ever writing any Rust yet — passing by at the moment, can't commit to fixing this myself.

jtagcat avatar Dec 07 '21 03:12 jtagcat

I would really love to be able to do this, but if I increase the number of posts in the feed it causes every single badly configured RSS reader (nearly all of them) to waste so much bandwidth constantly downloading the list of posts every 5 minutes. Would some kind of API access work instead?

Xe avatar Dec 08 '21 14:12 Xe

Would some kind of API access work instead?

By that, do you mean a separate/duplicate method/URL param/feed/? I'd be personally fine with it, but there's also a risk that that one is abused by the RSS clients as well.

An overengineered and log-collecting way to do this would be to track IPv6 or generic IP + user agent. Don't have many ideas other than doing it in the protocol.

JSON Feed supports next_url, but:

  • clients can still be braindead, and paginate fully every the time
  • clients might not implement it[^1]
  • JSON feeds are not widely supported

[^1]: though, the user can work around that by paginating, combining everything, serving the file to the client, and then changing the URL; but the client might differentiate the feeds then, as the source differs, yet most clients don't care


I'll once again remind you that I'm fine, and don't really benefit from this change.

It's a crappy bash script that crawls a list of URLs I give it (scraped) for titles, and creates .eml files with only title and URL, that are then mass-imported imported in to Thunderbird.
TB is currently the best one that supports opening URLs inside it (good for smaller posts, Twitter and YouTube)[^2].
This is so ugly it made me feel bad writing this, yet if I don't write a new app, there isn't an alternative what checks all the boxes.

[^2]: Also, bonus points for having feeds and items within be easily filterable and manually manipulatable..

jtagcat avatar Dec 08 '21 16:12 jtagcat

oh, neet, see RFC5005 mentioned in https://github.com/manton/JSONFeed/issues/129

jtagcat avatar Dec 08 '21 16:12 jtagcat

Blocked on #422

Xe avatar Jan 02 '22 03:01 Xe

This is unblocked. I will probably work on this on stream this weekend.

Xe avatar Mar 22 '22 01:03 Xe