lemon24
Gemini subscription support
https://gemini.circumlunar.space/docs/companion/subscription.gmi
Prerequisites:
- a way to fetch gemini:// URLs
- https://pypi.org/project/aiogemini/
- https://pypi.org/project/gemurl/
- https://github.com/kr1sp1n/awesome-gemini#programming
- https://github.com/cbrews/ignition
- https://framagit.org/bortzmeyer/agunua
- https://tildegit.org/solderpunk/AV-98 ("canonical" CLI client)
- https://tildegit.org/solderpunk/CAPCOM ("canonical" CLI feed reader)
- figure out how to handle TOFU
- a way to render GMI to HTML (re-check awesome-gemini above); only if we fetch linked entries
This might be a good way to explore how a plugin (PyPI) package would work.
I don't have time, nor a need for this at this moment. Closing. (Please reopen if you want to work on it.)
I created modules to retrieve and parse Gemini Feed.
https://git.xmpp-it.net/sch/Slixfeed/src/branch/master/src/slixfeed/retriever/gemini.py
I might consider to utilize aiogemini instead of Agunua.
https://git.xmpp-it.net/sch/Slixfeed/src/branch/master/src/slixfeed/parser/gmi.py
~It appears that urljoin does not work properly for Gemini URI scheme, albeit it does work with Gopher URI scheme.~
I did not know of gemurl, and I intend to try it.
https://github.com/raek/gemurl
Hi @sjehuda, thank you for looking into this! (based on the comment above, I'm assuming you intend to work on it, please correct me if I'm wrong)
Some broad suggestions on how to implement this:
-
There are two main components that need to be implemented:
- A retriever that knows how to take a Gemini URL and return a file-like object. Examples: http(s), file.
- If the underlying protocol has caching / conditional requests (analogous to HTTP ETag / Last-Modified / If-None-Match / If-Modified-Since), the retriever should return the relevant caching info after a request, and use it on the next request.
- A parser that knows how to transform the file-like object into feed + entry data. Examples: Atom and RSS using feedparser, jsonfeed.
- The benefit of decoupling retrieving from parsing is that a parser (e.g. Twtxt) can work with both the http:// and gemini:// protocols without having to know about them.
- A retriever that knows how to take a Gemini URL and return a file-like object. Examples: http(s), file.
-
These get registered with the high-level Parser object here; a plugin would call similar methods in its init function; examples:
- sqlite_releases registers a parser for a specific URL.
- twitter registers a retriever for a URL prefix and parser for a mime type (note the retriever and parser interfaces changed a bit since this plugin was discontinued, but the overall idea is the same).
-
I'd prefer this being an experimental plugin in the beginning; once it is stable enough, we can promote it to a built-in plugin (it should still be a plugin because of the extra dependencies).
Re. Agunua vs. aiogemini: Based on the number of releases (all other things being equal, more releases and more contributors is better), I'd prefer Agunua; aiogemini is OK too, although the retriever would need something like https://github.com/lemon24/asyncio-thread-runner to use it.
Re. URL normalization: The parsing logic is already doing some rudimentary normalization to ensure the appropriate retriever is found, I'd suggest not doing anything extra initially. (I see the current urllib.parse -based implementation does not make the host lowercase, but I guess that being allowed is protocol-dependent; regardless, can be improved at a later time).
Hi @sjehuda, thank you for looking into this! (based on the comment above, I'm assuming you intend to work on it, please correct me if I'm wrong)
I might agree to work on it, provided that I would have your guidance.
I might agree to work on it, provided that I would have your guidance.
Definitely, I'm available to answer any questions!
(No pressure, though; my assumption was in light of this comment saying this is not a personal priority for me (still the case); but I'll always prioritize supporting contributors.)