graspologic
graspologic copied to clipboard
graspologic-org
Combined, Versioned Documentation Site
Problem Description
We want to get to a place where our reference documentation and tutorial documentation is versioned, tightly linked, and easily displayable.
Secondary Issues
ReadTheDocs does not have the resources available at the free tier to consistently build both our tutorial documentation and our reference documentation (15mins and 3gb of memory, respectively).
We (or rather, I) made the decision that trying to cut down on our build time and memory usage was going to end up being ultimately a short term fix; as we add features and more tutorials, our build time is going to increase. We're going to have issues going forward and little fixes here and there are realistically only going to buy us a little time before we have future problems.
Current State
Right now we use RTD to generate our reference documentation. It generates our reference docs for every branch on our repo.
We use Github actions to generate our tutorial documentation. On merge to dev we automatically publish the one and only version of our tutorial documentation in a place that is navigable by a web browser. Our tutorials and reference documentation can't be tightly linked (in the sense that things may be relocated and we can't have anything in place that ensures they continue exist and we don't have any broken links). We also don't publish the latest release of our latest current final release - instead we publish what's on the bleeding edge by publishing dev only.
Requests of this Feature
- Allow us to build reference documentation (fast) or both reference and tutorial documentation (slower). The goal here is to be able to quickly validate our pydoc reference docs are building correctly with or without the tutorials being built. This may not be possible as we may be in a chicken and egg scenario, but it would be nice if we could say "do a full documentation build" as well as "do the fast reference documentation only build" for local development purposes.
- Each release - whether full release (merge to main) or pre release (merge to dev) should have a full documentation suite built from it, and this documentation suite should be published to a website of some sort for us to see.
- @bdpedigo edit - build previews linked in each PR like netlify used to do. These make reviewing code changes that affect docs much easier
As far as I can tell, we cannot build our documentation and have RTD host it as-is. Perhaps that is something we can do with another service?
Things we can explore:
- a graspologic documentation repository (meaning that the graspologic ci publishes built documentation to the graspologic-docs repo)
- other third party documentation publication services (GraSPy originally used netlify and we still technically have it building documentation for us - perhaps we use it in lieu of rtd? Can we own our documentation build via github action and just have netlify publish it for us? Can netlify let us "publish" these by version?)
@loftusa has been sad about our documentation and he may have some ideas as well, so I'm hoping he'll chime in here.
@bdpedigo, @j1c, @asaadeldin11 - anything else you'd like to add?
@dwyneprce I think we should explore the idea of having a graspologic repository as with time we will be adding new features to the repository and then documenting all that reference will again turn out to be a slow process.
GraSPy originally used netlify and we still technically have it building documentation for us - perhaps we use it in lieu of rtd? Can we own our documentation build via github action and just have netlify publish it for us? Can netlify let us "publish" these by version?
Yes, that is actually possible. Here is the link to a article I was reading on Medium.
https://marekpukaj.medium.com/build-with-github-actions-host-on-netlify-ebf5fa505616
I like this issue! I appreciate your multi-paragraph issue descriptions, Dwayne 🙂
I think (although am not sure) that we have the paid version of netlify, which might be nice given how much of a struggle building the documentation has become. I also haven't really dove deeply into the details of how things are currently set up on a technical level since we started doing splitting up the tutorials and reference documentation, so I'm not sure how parallelized things currently are?
I'm also wondering how expensive just getting the paid version of RTD is. Seems like it'd be within our budget, but again, not sure about the details of getting that set up.
I'm pretty agnostic about the technical details. I see time-to-build as not that huge of a deal, since we update dev branch infrequently enough that it's not a limiting factor - I'm more concerned about the outward-facing stuff (not having a link back to the main documentation page once you click the references, for instance, and the flat layout in the tutorials). It's been something I've been meaning to make a PR for myself, but haven't had the time.
I'm totally down with the documentation repo idea (we should for sure document that in the README if that's how we're doing it), and the netlify idea seems good as well. I'm also wondering if there's a way to track down which tutorials are using up the most processing bandwidth, and if we can target those specifically for trimming (although, like you mentioned, not a permanent solution)
I'm more concerned about the outward-facing stuff (not having a link back to the main documentation page once you click the references, for instance, and the flat layout in the tutorials). It's been something I've been meaning to make a PR for myself, but haven't had the time.
@loftusa I cannot understand this. can you explain a bit more if you have some free time.
I'm also wondering if there's a way to track down which tutorials are using up the most processing bandwidth, and if we can target those specifically for trimming (although, like you mentioned, not a permanent solution)
not sure on this. but there is a tool named lighthouse by google though which we can assess any web page's page experience and access valuable tips to improve its performance.
@loftusa I cannot understand this. can you explain a bit more if you have some free time.
yeah, see #665. I mean that the sidebar on the left here doesn't have subsections (the tutorials aren't grouped into "Model", "Simulations", etc)
and also that if you go here, then click "tutorials", there's no way to get back to the main page
@loftusa I propose to add a second level to the left sidebar which is an easy sphinx fix. I guess I had proposed it earlier too in some issue I don't remeber exactly which one though. Just we need to first organise the reference and the tutorial docs into proper subsections before implementing this fix.
Please let me know if you interested in this approach
Yeah, sounds good to me, just need to add a link back to the main page as well and then we're golden