jtv
Docs gone since 7.9
I noticed the docs went away going from 7.8.1 to 7.9. Now, the readthedocs page is empty. If this is intentional as the 7.9 docs are not complete perhaps it'd be best to default to 7.8.1 docs in the meantime so users know something exists.
No it's not intentional. But something changed in how buildthedocs builds worked, and so the existing configuration suddenly no longer worked. It was quite annoying.
I thought I'd fixed that. Wasn't trivial. But now it turns out that I still need to do more...
By the way the docs didn't fully go away - you can still search, and find class/function documentation!
It looks as if the indexes and the Markdown docs are gone.
That's a problem, we are using the docs heavily. Please fix them. Thank you!
I'll try, but I'm fairly ill right now. I think the documentation build logs are publicly accessible on readthedocs... If so, perhaps you can have a look and see if you can find the problem. They made some incompatible changes recently so the builds just suddenly started breaking, without any help from me.
Until then, all the documentation in there is also in the libpqxx source tree.
Also, you should be able to build the documentation on your own machine.
How do I do it?
It did not create a doc subdirectory when I built it.
I haven't done it in a while because Ubuntu's packaging of Firefox no longer lets me just view a file on the local filesystem. But it worked something like:
You may need some additional packages installed on your system - you'll have to dig around to find out which.
Configure a libpqxx build using the configure script. Pass the --with-documentation option on the command line.
Do a regular build. It should create the docs tree somewhere in the build tree. If it does not, that's typically a sign that one of those packages was not installed.
I've been trying various things by the way, but haven't succeeded yet at getting the readthedocs documentation to build properly again. :-(
Perhaps we should just go from the raw .md files and move it over to something like docusaurus? Or even a plan readme folder? @jtv
I was thinking I might go with mkdocs, but still on ReadTheDocs. But frankly I feel pretty demotivated at this point. Something made an incompatible change and I'm supposed to guess what it is.
Bisect what though? I don't know that I get to choose versions for all the components involved. Nor am I sure which components might be relevant, or which versions have been running.
It could be something simple, like how CircleCI seem to have just moved their generically named config file from a custom directory into the project root. Or it could be something that I've been doing wrong all along but that never caused quite enough trouble. At any rate it feels like I pretty much have to start from scratch.
One problem IIRC is that I couldn't find a reliable way to navigate the filesystem on the build slave.
I think I've made some progress. I can get ReadTheDocs / Sphinx / Doxygen / Breathe to generate documentation. But it's all a single page, with the introduction all the way at the bottom. I'll still need to find a way to control that.
Unfortunately the documentation for Breathe is driving me up the walls. It skips the basics and goes straight into the details. It also seems to contradict itself. I'm sure it's alright for people who already know and don't need the documentation, but...
@jtv FYI. Since Read the Docs basically uses the latest version library, it seems to be recommended that the version be fixed so that the document generation is not broken by the release of a new version.
It is recommended to use pip-tools to list the libraries and versions to be installed in requirements.txt.
See: https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html#pin-your-transitive-dependencies
I've sort of got documentation, but not in a nice way... It's all in one page, with the introduction all the way at the bottom.
Thanks for the tip about master_doc @tt4g — but weirdly, neither master_doc nor root_doc seems to do anything!
@jtv I have investigated conf.py and plugins since then, but have not found the cause.
It seems difficult to pinpoint the cause since several libraries are used for document generation.
Well in a disturbing way I'm glad it's not just me, thanks. If only we had better documentation... I may have to search for examples instead of the official docs.
I used the ReadTheDocs Docker image to investigate what was going on.
As it turns out, prior to version 7.9, Doxygen generated documents were hosted on ReadTheDocs.
Doxygen generates XML and HTML documents in doc/xml and doc/html.
When I accessed https://libpqxx.readthedocs.io/en/7.8.1/, I saw doc/html/index.html generated by Doxygen.
It seems that doc/html/index.html was being displayed because the Doxygen generated HTML file was copied as a ReadTheBox document by html_extra_path.
https://github.com/jtv/libpqxx/blob/21afb0d6ec3d31424df6b7aac04c6b20c55c0a65/doc/conf.py#L135-L139
However, since version 7.9, documents generated by Sphinx from doc/index.rst seem to appear instead of doc/html/index.html.
I think the internal behavior of Sphinx has changed so that index.html generated from index.rst overwrites doc/html/index.html copied by html_extra_path.
Unfortunately, since Breathe does not seem to be able to generate the same index.html as Doxygen, it seems better to consider hosting Doxygen-generated documents in ReadTheDocs without using Sphinx.
@tt4g if it works, at this point I'll gladly ditch Sphinx, ReST, and Breathe. I don't even remember at this point why that was supposed to be so much better. But the docs for readthedocs aren't too clear on whether you actually need Sphinx. In one place it seems to say that you do, and then in another place it seems to say that mkdocs is an alternative, and in that case there may be room to roll your own...
I've started #832 to just try again from scratch.
@jtv Yes, the problem would be solved if the assets generated by Doxygen were hosted as is. I couldn't try to build with Doxygen alone without Sphinx on ReadTheDocs, so I had to end my investigation there.
Oh but there's probably a way to include existing HTML files into the Sphinx output...
Sphinx provides the ability to import existing HTML files by specifying html_extra_path in conf.py: https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-html_extra_path
Previously, index.html generated by Doxygen took precedence over index.html generated by Sphinx (which is generated from index.rst). But now it seems that Sphinx generated index.html takes precedence.
I checked for other ways to import existing HTML files, but could not find one.
OK, I am now working along the lines of "we'll pretend to use Sphinx, but really I'm just abusing its config file as a launcher for Doxygen." It's a silly setup, but RTD has a feature in beta where you can just run your own build process to generate HTML however you like. That looks attractive, once it leaves beta.
I've got something working with readthedocs! \o/
It's not exactly pretty, but it's functional. Really uses only Doxygen, not Sphinx or Breathe or Exhale. See what I've got built for my PR, while that build lasts. (I assume it'll get cleaned up at some point, since it's for a PR.)
It's a complete rewrite though, and I gave up the local documentation build.
I don't know whether anyone out there builds their own documentation for libpqxx... Bringing it back now shouldn't be too hard.
I haven't merged this change yet. I'll need to see about reintroducing the version string; that was the whole reason I had to pre-process the Doxyfile IIRC.
I'm trying to bring back the local documentation build. First in the configure setup.
I think I've got it all working again, more or less. Still a frightful amount of error output from Doxygen, and I have yet to pin the versions. May not be so urgent though, now that I'm basically bypassing Sphinx and Breathe and all that.