sphinx
sphinx copied to clipboard
sphinx-doc
LaTeX: support for booktabs-style and zebra-striped tables
This transfers here development from #6666 to continue it and merge with #6671.
- [x] revamp initial work of @sephalon in #6666 for compatibility with current Sphinx LaTeX and to address initial issues. This has been done originally at #6666 PR but I am moving it here and will adapt WIP status until the whole thing is done.
- [x] merge initial work of @sephalon from #6671
- [x] allow some tables to be styled the traditional way in documents globally using the booktabs option
- [x] allow tables to be styled the booktabs way in documents globally not using the booktabs option
- [x] as for booktabs style, think about locally styled tables; LaTeX aspects are not complicated based upon booktabs-styling with #6666 as merged here, the problem to obtain as light-weight mark-up as possible. Simplest seems to go via classes but then we must used
latex-prefixed names to dispel user expectation that it also applies to say html build.Actually using a container directive is possibly better as on LaTeX side it will wrap in a scope limiting environment. Maybe for first installment we can simply advice using some such recipe (container directive plus raw latex) which does not need extra code in Sphinx itself.
finally I used classes with no latex- prefix even though they also may influence html output, but this sounds actually good
removed longish paragraph which was in part obsoleted and too complex to edit; in brief all changes are opt-in except that
-
for merged cells in a single row the row colour (or the dedicated merge row colour) is now obeyed, so people who added manual usage of
\rowcolorto their documents had only white merged cells but now the merged cell in single row (only) will react to row colour. -
for people who added manual usage of
\rowcolorscommand to their document, the effect will be overwritten by this PR. Sphinx does not itself use\rowcolorsdue to the latter limitations (and the fact that not all people have xcolor). Sphinx intervenes via its table templates and own code to influence the row colours, and this will overrule effect of manually inserted\rowcolors. I don't know if this means the PR should go to master rather than 5.x.
Memo: https://github.com/sphinx-doc/sphinx/pull/10759/commits/bb859c669679baebd8cc8d10c99382478c0d1647 probably closes #8220 as it makes mention of rst-class more prominent and links to it.
In order to illustrate the effect of this PR I am adding here snapshots from our own document, using:
latex_table_style = ['booktabs', 'colorrows']
and the default colours activated by 'colorrows':
first table
another
Now a long table with a tabularcolumns directive.
on next page:
The effect was obtained via this
>{\sphinxcolorblend{!95!red}\centering\noindent\bfseries\color{red}}\Y{.12}
as configuration for the third column. It works here because my TeX installation has the LaTeX xcolor.sty. On user devices with xcolor.sty not installed (I don't know actual situation but many years ago this was not automatic in Linux distros), the \sphinxcolorblend will be ignored but trigger a warning at very end of latex build console output (and also at very end of pdf after indices).
Now a complex grid table with a custom tabularcolumns (with no vertical lines)
.. tabularcolumns:: >{\sphinxcolorblend{!90!blue}}TTT>{\columncolor{yellow}\sphinxnorowcolor}TT
and booktabs style
.. rst-class:: booktabs
It illustrates various features. The conf.py is:
latex_table_style = ['colorrows']
latex_elements = {
'preamble': r'\usepackage{booktabs}',
'sphinxsetup': '''TableRowColorOdd=red!10,
TableRowColorEven= blue!10,
TableRowColorHeader= {gray}{0.75},
TableMergeColorHeader=green,
TableMergeColorOdd=green!20,
TableMergeColorEven=green!10''',
}
where the colours have been chosen at random (and test the new acceptance of \colorlet-like syntax). Also r'\usepackage{booktabs}' was added to the 'preamble' of latex_elements as the document does not use globally 'booktabs' in latex_table_style but this table requires it (else, there would be a warning at very end of the latex build console output and at very end of pdf after indices).
One sees in this table:
- merged cells remain with no background colour, except for the case of single-row multi-column cells. They then use the special Merge color, if set.
- in the header each row uses the same colouring style
- the fifth column displays the normal colours corresponding to
'sphinxsetup' - the fourth column obeys yellow column color thanks to
\sphinxnorowcolor - the first column uses
\sphinxcolorblend, whose effect is ignored in merged cells.
grid table
----------
.. tabularcolumns:: >{\sphinxcolorblend{!90!blue}}TTT>{\columncolor{yellow}\sphinxnorowcolor}TT
.. rst-class:: booktabs
+---------+---------+---------+---+--------+
| header1 | header2-3 | a | normal |
+---------+---------+---------+---+--------+
| header1 | header2-3 | a | normal |
+---------+---------+---------+---+--------+
| header1 | header2-3 | a | normal |
+=========+=========+=========+===+========+
| cell1-1 | cell1-2 | cell1-3 | normal |
+---------+ +---------+---+--------+
| cell2-1 | | cell2-3 | normal |
+ +---------+---------+---+--------+
| | cell3-2-par1 | d | normal |
+---------+ +---+--------+
| cell4-1 | cell3-2-par2 | e | normal |
+---------+---------+---------+---+--------+
| cell5-1 single-row merged | f | normal |
+---------+---------+---------+---+--------+
The latest (now squashed) commits in this PR fix some issues with merged cells which were not yet documented as such: when a tabularcolumns directive uses no |, nevertheless merged cells would sometimes use them; also, horizontal widths was not computed correctly (but the discrepancy was not really visible). On the other hand if the tabularcolumns directive contains a |, nothing changes, i.e. merged cells are bordered too vertically, even though in complex column spec one could imagine only some columns are bordered. In theory, the LaTeX writer could analyse where the | are located in the colspec and use this to influence its handling of merged cells, but this should be topic of another PR: this one only takes care of no | at all in the tabularcolumns colspec.
again one of those transient test_build_linkcheck.py failures... https://github.com/sphinx-doc/sphinx/runs/7821615851?check_suite_focus=true which are not related to this PR...
There is a quite strong deficiency of LaTeX's \cline. It shifts a bit the text upwards for cells underneath it. One can exaggerate the effect by setting \arrayrulewidth to a high value like here 3pt:
(see the two leftmost red "circles")
edited: Sphinx now has workarounds for those \cline problems; this went through some added mark-up in the table for "correcting" measures; in particular the main well-known problem that LaTeX \cline's is hidden behind cell colour panels has been solved in Sphinx domain.
The other "feature" (the same actually) one sees clearly (the red "circles" on the right) is that colour panels cover it. This is mentioned in colortbl documentation, but the recommendation to use \hhline seems impractical to use due to the complex syntax of \hhline. The \cmidrule of booktabs do not have this problem and we could let \cline act like them, but they look appropriate only in tables with no vertical lines, as they will create gaps in these vertical lines!
You may think such basic problems have been solved years ago in LaTeX... but alas no. The alignment preamble itself is NOT available from inside the table cells in an easy to work with form! Incredible but true. (they are many incredible but true limitations in LaTeX).
For example one gets this for the above table.
> \@preamble=macro:
->\ialign \bgroup \unhcopy \@arstrutbox {\vrule width\arrayrulewidth }\hskip \c
ol@sep \setbox \z@ \hbox \bgroup \bgroup \d@llarbegin \sphinxcolorblend {!90!bl
ue}\ignorespaces \@sharp \unskip \relax \d@llarend \egroup \egroup \begingroup
\CT@setup \CT@row@color \CT@cell@color \CT@do@color \endgroup \@tempdima \ht \z
@ \advance \@tempdima \minrowclearance \vrule height\@tempdima width\z@ \unhbox
\z@ \hskip \col@sep {\vrule width\arrayrulewidth }&\hskip \col@sep \setbox \z@
\hbox \bgroup \bgroup \d@llarbegin \ignorespaces \@sharp \unskip \relax \d@lla
rend \egroup \egroup \begingroup \CT@setup \CT@row@color \CT@cell@color \CT@do@
color \endgroup \@tempdima \ht \z@ \advance \@tempdima \minrowclearance \vrule
height\@tempdima width\z@ \unhbox \z@ \hskip \col@sep {\vrule width\arrayrulewi
dth }&\hskip \col@sep \setbox \z@ \hbox \bgroup \bgroup \d@llarbegin \ignorespa
ces \@sharp \unskip \relax \d@llarend \egroup \egroup \begingroup \CT@setup \CT
@row@color \CT@cell@color \CT@do@color \endgroup \@tempdima \ht \z@ \advance \@
tempdima \minrowclearance \vrule height\@tempdima width\z@ \unhbox \z@ \hskip \
col@sep {\vrule width\arrayrulewidth }&\hskip \col@sep \setbox \z@ \hbox \bgrou
p \bgroup \d@llarbegin \sphinxnorowcolor \ignorespaces \@sharp \unskip \relax \
d@llarend \egroup \egroup \begingroup \CT@setup \CT@color {yellow}\@tempdimb 6.
0pt\@tempdimc 6.0pt\relax \CT@row@color \CT@cell@color \CT@do@color \endgroup \
@tempdima \ht \z@ \advance \@tempdima \minrowclearance \vrule height\@tempdima
width\z@ \unhbox \z@ \hskip \col@sep {\vrule width\arrayrulewidth }&\hskip \col
@sep \setbox \z@ \hbox \bgroup \bgroup \d@llarbegin \ignorespaces \@sharp \unsk
ip \relax \d@llarend \egroup \egroup \begingroup \CT@setup \CT@row@color \CT@ce
ll@color \CT@do@color \endgroup \@tempdima \ht \z@ \advance \@tempdima \minrowc
learance \vrule height\@tempdima width\z@ \unhbox \z@ \hskip \col@sep {\vrule w
idth\arrayrulewidth }\tabskip \z@ \cr .
<recently read> \@preamble
fomr inside the table. Which is some translation of the table preamble. Notice some things are expanded like \CT@column@color but not others. But imagine you are in a cell, how to you know which part of the above applies to it? LaTeX has no easy answer to that. Anyway, for the LaTeX macro programmer that's it. No wonder we do not have LaTeX packages solving basic problems of table styling in a coherent manner. They all play an artistic dance on the top of a needle.
The last paragraph in small typeface is about my psychological preparation before I solved the above mentioned \cline-induced problems in a somewhat different manner than the one mentioned there.
On the other hand Sphinx LaTeX writer does have the preamble and may thus inform, say for example its own \cline of where \vline's are expected or not. Thus perhaps using the \hhline but I don't feel like modifying our existing mark-up and have to update again all tests. I see a way with some LaTeX coding via a suitable fake line which would serve to put in place the \vlines fomr the preamble and add horizontal rules in given cells, \@arstrutbox has to be hacked aways, colour matters too, and... after starting to code it, the problem is I need to know how many columns the table has! and believe it or not this information is not easily gathered so this line of thought is leading to adding more mark-up which is what I wanted to avoid... (the number of columns is the number of \CT@setup in \@preamble but LaTeX has no way to count things, we have to code it; and who knows how that could interfere with some other LaTeX package?... nevertheless maybe I will do this "better" \cline)
The problems with LaTeX \cline being hidden behing colour panels and reducing vertical space are solved. The figure uses a high value of \arrayrulewidth to better check all is correct.
The above with normal borders:
Try to do that with "vanilla LaTeX" in an automatic way... (and I demand vertical spacing is coherent and lines not covered by colours...)
The major problem, not for this PR, is how to allow complex multi-row multi-column cells to receive a colour? Because the colour panels are drawn from top to bottom, this means the text must be printed from bottom row, else it will be underneath the colour panels. But if one uses \cellcolor at that location it is too late to know the colour in the top rows. For manually done documents this is not a problem as one can add manually the colours, and use \multirow* to place contents at bottom (but \multirow* has limitations and requires manual adjustments depending on contents). As commented in the Sphinx latex code, it seems the way is to leave the latex cell contents on top but store this for later usage only. The reason colour can be obeyed in single-row multi-column cell is that the actual contents are in fact evaluated in last cell, because Sphinx via macro-coding moves it. But this means that a \cellcolor in these contents would act only on last cell; anyway it is deactivated. And similar macro-coding to insert manually & works in a row but can not work to shift to bottom of a multirow. (I am leaving these comments for future maintainers... including myself as I will forget).
Anyway, I think I am now done with this never ending PR. (I already squashed 28 commits into one, and 12 are left...).
Final thing I may add is to fill the gap in booktabs style with the correct colour. But as said in the docs of the added confval latex_table_style, I feel these gaps are good. Once one colorizes them, one wonders why keeping horizontal rules at all, so one is tempted by borderless+colorrows style rather than booktabs+colorrows. So I wait if someone asks for it.
I have added in my clone at https://github.com/jfbu/sphinx/commit/abab0bf1e43d8ce809bc62452f58930b628731a3 an extra commit in order for booktabs + colorrows to give tables with no "white gaps". Here are some snapshots of what it produces in our own document:
edit: I took the snapshots at a time the "night-shift no blue waves" had already ticked-in... so this might change a bt the colours...
This one has a problem because the small gap underneath the top rule is filled with the default colour for headers (slighly darker gray than the one for odd rows). To fix this is a bit tough on macro side because it will need \sphinxtoprule to peek ahead to see if next token is \endhead, \endfirsthead, or \sphinxtableatstartofbodyhook which is only way to realize there is no header and thus adapt the colour. Easier would be to add mark-up to the table to signal to LaTeX macros that it has no header.
Also this one has a problem, because we see that the column color tint does not appy. For it to apply we would have to use an altogether different approach than using the \specialrule of booktabs.
Besides globally on zooming on pictures I think I am seeing anti-aliasing which might be a sign there is some extra space I did not take into account, andI would need to dig into \specialrule code from booktabs which I don't really feel like doing.
So this is bit problematic (when I started writing this comment and took the snapshots I was not aware of all problems).
update 1: the problem with colour for tables with no header has been fixed at https://github.com/jfbu/sphinx/commit/765ccda6f0a5001f61ea6d5b1d68f3f7af0198f4. And regarding zooming on the pictures I should rather have zoomed on the PDFs and this looks fine although it seems either my eyes or the PDF viewer does anti-aliasing at intermediate zoom levels but this depends on colours. to be honest, zooming at high levels on tables reveals the effet of the \sphinxcolorpanelextraoverhang parameter whose default is 0.1pt which makes the colour background extend a tiny bit more to left and right than for the filled gap and the horizontal rules, perhaps I should propagate this extraoverhang to the width of the rules... (this thing never ends)
One wonders after colorizing the gaps why keep the black rules at all... and why LaTeX produces generally so cramped tables. (LaTeX's \arraystretch is hopeless but works for simplest cells with single-line contents).
update 2: I have continued at https://github.com/jfbu/sphinx/tree/latex_booktabs_nowhitegap to polish some details. However there is a problem for which I feel there will not be easy solution (probably needed to hack longtable): the \midrule is part of the longtable headers so its action is frozen and decided when longtable parses until \endfirsthead, \endhead. So the colour used until the midrule will be always the "RowColorOdd" one. But on top of next page the colour of first non-header row depends on parity one can not know it in advance. The probably easiest way to fix would be to hack longtable so that when on top of a new page body rows always start with "odd" colour, rather than continuing the alternance from bottom of previous page. Or one simply accepts the situation. Usually the "RowColorOdd" should match not too bad with the "RowColorHeader" anyhow but it might contrast a bit with "RowColorEven". Not sure I want to solve this (I was supposed to have finished this PR something like 10 days ago...). And there is a somewhat similar problem with the bottom rules... (they come from \endfoot and choose the colour when longtable first parses them I think; must check the longtable code again).
edited (of course...)
In one of the longtable snapshots of https://github.com/sphinx-doc/sphinx/pull/10759#issuecomment-1214189134, one sees that header colour is applied to the table continuation hints.
But this was an oversight. In first phase of development of this PR I noticed things such as:
when testing with \rowcolors command (in its case, this applied even to the table caption, not only to continuation hints as in this picture). So I dropped it and initially did not add row colours to the headers. They were other problems, then these problems were solved and I re-added row colours to header rows, and forgot about image above because testing only with wide long tables whose width was larger than the continution hints...
With the extra patch now pushed one obtains for example on top of a continuation page in our own document:
and at bottom of a page for the same long longtable of Deprecated APIs:
About this question from earlier version of this comment:
I guess this removal of the header row colour from contination hints should be default?
It has to be not only the default, but with no added customizing option.
I have now merged the separate work on colorizing booktabs gaps mentioned in earlier comments so that it is done per default and tables look as in https://github.com/sphinx-doc/sphinx/pull/10759#issuecomment-1218424121. I consider now it is better indeed to colorize these gaps if booktabs+colorrows is chosen by user conf.py, and the longtable aspects should not be considered blocking. However I needed to make one extra addition to the longtable template. finally I moved the change away from the longtable template
This brings to an end (? I was already saying PR was ready 10 days ago...) my work on this... as my Sphinx+LaTeX time will now be more scarce in coming months I wanted to finish a coherent whole.
Two short notes:
- in the Python code I used
colsepto designate either''or'|'. The name comes from initial #6666. It looks natural as shorthand for "column separator". But LaTeX uses\tabcolsepfor the dimension register denoting whitespace (on each side of|). So the name is a bit surprising to LaTeX user. I didn't look for a better one. - recently there is a new LaTeX package https://github.com/lvjr/tabularray which looks promising. It may in future help to replace all three of tabular, tabulary and longtable perhaps (I notice it uses optionally internally varwidth package as Sphinx does via explicit mark-up). But this can not be until a few years because it is undergoing active development (first release in May 2021, see its ChangeLog, with some breaking changes). By the way it uses a
colsepkey in its table config with the meaning afaict of interfacing to some equivalent of\tabcolsepillustrating the terminology problem mentioned in previous item.
@AA-Turner Although the various comments I added when I was actively working on this may give a frightening impression, this PR is completed and I am not planning any further change... it adds opt-in features only (some are tested in our own document via a slightly modified conf.py). We can wait to see if @tk0miya has comments. For 6.0.0 release I would like to make some configuration the default, so that tables look better in LaTeX.
If no objection I will merge this in 5.x in a few days, so that it makes it to 5.3.0. The changes are opt-in only. For 6.0.0 I will propose probably to make some of it the new default for rendering tables in LaTeX.
Edit: In view of #10896 I will merge this (if no objection) tomorrow October 6th, around 8PM European time ;-) Ah edit again, release is actually planned for the next week=end, so I will wait until Wednesday October 12th...
now merging a few hours earlier than announced due to personal constraints
alea jacta est ;-)
real activation will be for 6.0
@jfbu Thank you very much for your hard work to bring these features upstream! :-)
@sephalon sorry I messed up in the squashed merge and the git commit lost your co-authorship... and I can not force push to 5.x...