[WIP] Move documentation to RST/readthedocs

[hainest]

Improvements:

• All documentation is written in a single format
  • That format is not LaTeX
  • RST is used by many large projects, including the Linux kernel
  • Uniform look-and-feel
• Docs are automatically exported to readthedocs.io via GitHub actions
  • Also automatically versioned
    • This makes doing releases much easier
  • We can point users to stable web pages instead of PDF section numbers in a commit
  • Docs can still be rendered to PDF or LaTeX
• Types and functions are fully linked (via hyperlinks in html or links in PDF)
• Types and functions can now be referenced across toolkits
• All docs are uniformly searchable across toolkits
• Documentation has been split between user interfaces and developer ones
• All example code is now in the dyninst/examples repo and built on every Dyninst PR
  • Example code rendered in the docs automatically tracks changes in Dyninst
• Code coverage is improved by a lot
  • I can't really measure, but I'd say an order of magnitude, at least
  • Information previously hidden in source files is now in the same place as the rest of the docs

---

TODO

• [x] Is the navigation pane usable?
• [ ] Are the overview/userAPI/dev API sections laid out reasonably?
• [x] 'Usage' versus 'Example' code snippets
• [ ] Missing elements?
• [ ] Improve instructions for writing documentation. RST is not intuitive, but I tried very hard to make everything uniform so that adding new stuff is effectively copy/paste/modify.
• [ ] Add CI (I'll do this).
• [ ] Add page for runtime debug environment variables

Fixes #394 Fixes #711 Fixes #800 Fixes #1255 Fixes #1260

[hainest]

@kupsch @bbiiggppiigg @rochauha Could you take a look at the TODO items in the description and provide feedback if/when you have time? The instructions for building are in docs/README.md.

[bbiiggppiigg]

So what I should do is build the docs, and then check if the TODO items can be checked off based on the built documentation ?

[hainest]

So what I should do is build the docs, and then check if the TODO items can be checked off based on the built documentation ?

Yup. You don't have to necessarily go through each TODO item. I'm really looking for feedback on the usability of the docs in general.

[bbiiggppiigg]

So what I should do is build the docs, and then check if the TODO items can be checked off based on the built documentation ?

Yup. You don't have to necessarily go through each TODO item. I'm really looking for feedback on the usability of the docs in general.

I checked that I can build the html doc, and the navigation seems fine to me.

For the layout, I don't know if it would be better for the developer docs for an API to be directly below the corresponding toolkit API. Even though the separation of developer doc from toolkit API make sense, the developer docs on its own looks kind of arbitrary (Maybe because the file are named kind of arbitrarily).

Also, I thought the documentation should be centric around C++ classes, but right now it seems to be centric around files, which I'm not sure if it is an issue or not.

I don't know too much about writing RSTs. This PR is too huge that my chrome will hang when I navigate the files changed tab.

[hainest]

So what I should do is build the docs, and then check if the TODO items can be checked off based on the built documentation ?

Yup. You don't have to necessarily go through each TODO item. I'm really looking for feedback on the usability of the docs in general.

I checked that I can build the html doc, and the navigation seems fine to me.

Roughly how long did it take to build on your machine? I forgot to include a patch to enable parallel builds, so I'll do that.

For the layout, I don't know if it would be better for the developer docs for an API to be directly below the corresponding toolkit API. Even though the separation of developer doc from toolkit API make sense, the developer docs on its own looks kind of arbitrary (Maybe because the file are named kind of arbitrarily).

I took the hint from other readthedocs pages on separating them into different sections.

Also, I thought the documentation should be centric around C++ classes, but right now it seems to be centric around files, which I'm not sure if it is an issue or not.

Funny enough, this is something I wanted to change during this migration. I just completely forgot about it as I slogged through the existing "documentation". I definitely agree we should do this.

I don't know too much about writing RSTs.

Yeah, I need to include a small example with links to the relevant RST docs.

This PR is too huge that my chrome will hang when I navigate the files changed tab.

No surprises. GitHub's interface is ass.

[bbiiggppiigg]

Now I think about it, a page for runtime debug environmental variable can really help.

[hainest]

Now I think about it, a page for runtime debug environmental variable can really help.

That's a good idea. Added to TODO.

[hainest]

@kupsch I'm still looking at getting the latex to build. There seems to be certain permutations of directories that cause it to fail. I'm trying to find the minimum set to see if I can narrow it down to a particular file. Based on what I could find in forums, it seems like having circular references in the latex generated by sphinx doesn't work very well. I have a suspicion that may be what's going on here.

[hainest]

@kupsch After more experimenting, it really is the case that the generated tex file is too big to be parsed by pdflatex. I found that using luatex worked.

make latex
cd _build/latex/
make LATEXMKOPTS="-lualatex"

The navigation pane for the resulting PDF is terrible, but I can't figure out how to make RST create the same structure in PDF as it does for HTML. One problem at a time, I guess.

As an aside, I've had no problems using parallel builds by enabling parallel_write_safe in env/lib/python3.10/site-packages/sphinxcontrib/remoteliteralinclude.py. Then you can use make latex SPHINXOPTS='-jN'.