openzim
Create/publish the user documentation
At this stage this seems necessary to publish a user documentation. Https://Readthedocs.com seems a good candidate.
I'm not sure that we need to transform existing documentation. Adding a paragraph on all scrapers README with a link pointing to existing one would be sufficient from my PoV.
@benoit74 Your link is not the scraperlib documentation. Explaining the goals, the paradigm of the scraperlib and the dev facing API of the scraperlib is the puropose of this ticket.
I don't think this is the correct way to address what I believe the underlying issue is. What we need is a general Make/Contribute-to/ a Scraper documentation. This would fit with the other documents referenced above and would be actually useful while a standalone scraperlib doc won't be of much help to do so.
Sure we could gen the auto-doc off the source code but the value vs just browsing the code on Github would be minimal. Just afraid this would close this ticket without helping onboarding.
@rgaudin Not 100% sure I get you right. But I can only disagree with a comment which in a nutshell says "no need to document library API, devs who want to use it should read the code".
For the rest, general (cross repo) documentation can have a place in the "overview" repo. But, how to use the scraperlib does not match this definition IMO.
Lets discuss this in detail at the next meeting.
This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.
![stale[bot] avatar](https://avatars.githubusercontent.com/in/1724?v=4 "Published by a pub.dev verified publisher"){kind=small}
As someone who just created a new scraper, here are my thoughts:
- It was difficult for me to understand what capabilities existed in the scraperlib versus what I was expected to build.
- It wasn't clear to me when the documents/code here, the GitHub wiki, or the OpenZim wiki was preferred.
I like automatically updating docs based on current source code e.g. GitHub pages triggered on release/merge because it's a faster way for me to understand the shape/intent of a library without diving into the details.
I like automatically updating docs based on current source code
We all do I believe and I think that scraperlib would be a good candidate! Do you want to take a look at it? I think we mostly need this setup, with the appropriate actions and the pointers to the right docstring convention.
We'd host it at readthedocs I think, as we do for other projects.
I believe libzim uses sphinx but I have also used mkdocs in the past and I find it easy to configure, flexible, readable and good looking. It's an active and popular tool.
Here's a good example of config for it: https://github.com/pawamoy/aria2p/blob/main/mkdocs.yml
@rgaudin sure, this is something I can look at probably in four-ish weeks assuming the issue is still open.
I've got a bit more to do with the DevDocs scraper to improve the UX after the release, once that's done I can scoot over to this assuming no other show-stoppers.
I've published the documentation at https://python-scraperlib.readthedocs.io/en/latest/
Obviously, only current version is published because it misses the proper mkdocs code / configuration to build doc for former version. I don't think it is worth it / possible to invest time in publishing doc for previous versions. Once a new version is published, it should start to pin these documentation (in addition to "latest" which we already have).
Can you please review online website and advise on what is left to do from your PoV?
Aside from fixing docstrings so that everything fits nicely, I don't see much "needed" changes. Probably many things can (and hopefully will) be fine-tuned, but nothing mandatory so far.
This is great 🚀 I've looked at it briefly and I think the overall doc is OK. Content is beyond the scope of this and will be improved overtime.
I suppose this is enough for now then, since there is no more feedback, let's close this issue as done