lemon24
Twtxt support
Twtxt is a text file which may be hosted over RSYNC, FTP, Gemini, Gopher, and even HTTP.
It is simple to parse.
def twtxt(content: str) -> dict:
feed = []
for line in content.split('\n'):
if line and not line.startswith('#'):
line_split = line.split(' ')
entry = {
'content' : line_split[1],
'date' : line_split[0]
}
feed.append(entry)
return feed
I wrote a module at https://git.xmpp-it.net/sch/Slixfeed/src/branch/master/src/slixfeed/parser/twtxt.py
Resources
- https://twtxt.dev/
- https://github.com/buckket/twtxt
- https://codeberg.org/sch/delightful-twtxt
- https://github.com/snarfed/granary/issues/908
- gemini://woodpeckersnest.space/~schapps/journal/2024-08-02-good-news-friday-a-new-dawn-of-syndications.gmi
Hi, I'm open to having this in reader, but I'm focusing on other things at the moment.
If you want to implement it, I've left some general guidance in https://github.com/lemon24/reader/issues/296#issuecomment-3455453681 (HTTP should work out of the box, and Gemini should work once a retriever is implemented).
A possible problem with Twtxt is that AFAICT it doesn't have a MIME type mandated by specification, and the parser selection logic relies on the MIME type to figure out what parser to use before the content is actually read.
A "local" solution would be to have a custom retriever for twtxt+<proto>:// that delegates to the HTTP/Gemini retriever and just sets a MIME type the parser is expecting (e.g. text/x.twtxt; for example, the discontinued twitter plugin did this).
Not sure what a general solution would look like – either a formalization of the <format>+<protocol>:// scheme above, or some way to sniff the content type; anyway, this is not a blocker.
