longtable not compatible with 2-column LaTeX documents

[nrnrnr]

For some time I have been using pandoc to create two-column PDF documents via LaTeX. (This can be accomplished with some abuse of the fontsize variable.) However, in the transition from 1.9.x to 1.11.1, all tables appear to have become longtable environments, and when presented with two-column format, longtable falls over:

pandoc: Error producing PDF from TeX source.
! Package longtable Error: longtable not in 1-column mode.

See the longtable package documentation for explanation.
Type  H <return>  for immediate help.
 ...                                              

l.267 \begin{longtable}

I see in past issues that the transition from ctable to longtable was decided on a while back, but I really hate to lose the ability to create two-column documents, which I use heavily. I also understand the issue of wanting to stick with relatively standard LaTeX packages.

Is it possible that the old ctable back end could be resurrected via some kind of command-line option? It sounds horrible, but I can't think of a better solution.

[nrnrnr]

Regarding the possibility of using longtable in 2-column documents, I found this thread on a Stack Exchange site, but it is not promising: http://tex.stackexchange.com/questions/39686/longtable-alternative-for-twocolumn-documents

[scmbradley]

I have a related issue that probably isn't worth opening another issue ticket for. Rather than a "resurrect ctable" solution, I'd prefer an option that just turns tables into very basic tabular environments. Or maybe a new style of markdown (or some sort of in-text flag) that outputs basic tabulars.

[scmbradley]

As a workaround, one can pandoc into tex and then use sed (or whatever) to replace all instances of longtable with tabular or ctable or whatever other environment you want...

[nrnrnr]

@scmbradley is it really that simple?

[dsoto]

@nrnrnr @scmbradley From what I can tell, if you use tabular, you also have to add table environments. I can do this manually, but I don't see a way to do it with the template.

[daroczig]

It would be indeed very useful if ctable, longtable or other similar environments would not be hard-coded, but the user could pass it e.g. via a pandoc option.

[terceiro]

I solved this by using this tip: replacing longtable with supertabular made it work for me.

[dtinth]

When replacing longtable with supertabular, you may get error about undefined command \endhead. Simply remove that from the LaTeX source.

To center the table, wrap them between \begin{center} and \end{center}.

Here's the code I add to the build process (in Ruby). result is the conversion result from Pandoc (actually generated from ipython nbconvert):

result.gsub!('\begin{longtable}', '\begin{center}\begin{supertabular}')
result.gsub!('\endhead', '')
result.gsub!('\end{longtable}', '\end{supertabular}\end{center}')

[nvasilakis]

Hey everyone, sorry for resurrecting this one -- do we have any way of resolving this beyond using sed? In theory, it should be able to just amend the latex template -- but the latex template does not seem to allow edits to the table structure. Any ideas?

[jgm]

If there were no other differences, you could do a renewenvironment in the template, and redefine longtable as table. E.g.

\let\longtable\table
\let\endlongtable\endtable

But there are some differences between the two environments, e.g. as regards handling of footnotes, so this probably won't work in general. (It might work sometimes.)

+++ Nikos Vasilakis [Aug 17 15 20:32 ]:

Hey everyone, sorry for resurrecting this one -- do we have any way of resolving this beyond using sed? In theory, it should be able to just amend the latex template -- but the latex template does not seem to allow edits to the table structure. Any ideas?

— Reply to this email directly or [1]view it on GitHub.

References

1. https://github.com/jgm/pandoc/issues/1023#issuecomment-132058854

[nvasilakis]

Thanks John! This does not work though, I think because the generated table code has a number extra stuff -- for instance: ! LaTeX Error: Unknown float optionc'.`.

However, I am now thinking: wouldn't a solution based on detecting supertabular work? For example, in LaTeX.hs source (intended mostly as pseudocode -- my haskell is rusty):

  return ( if ("supertabular" `isInfixOf` (writerTemplate opts)) then
           ( "\\begin{supertabular}[c]" <>
                braces ("@{}" <> colDescriptors <> "@{}")
           $$ capt
           $$ headers
           $$ vcat rows'
           $$ "\\end{supertabular}"
          )
         else -- normal old behavior
           ( "\\begin{longtable}[c]" <>
                braces ("@{}" <> colDescriptors <> "@{}")
                -- the @{} removes extra space at beginning and end
           $$ capt
           $$ "\\toprule"
           $$ headers
           $$ endhead
           $$ vcat rows'
           $$ "\\bottomrule"
           $$ "\\end{longtable}"
           )
         )

[jgm]

I don't really like making behavior sensitive to what's in the template in the way you describe.

What about doing what I suggested, but with supertabular instead of table? Does that work? Note that the midline, topline, bottomline are from booktabs and should still work with supertabular, I think.

+++ Nikos Vasilakis [Aug 17 15 23:23 ]:

Thanks John! This does not work though, I think because the generated table code has a number extra stuff -- for instance: ! LaTeX Error: Unknown float optionc'.`.

However, I am now thinking: wouldn't a solution based on detecting supertabular work? For example, in LaTeX.hs source (intended mostly as pseudocode -- my haskell is rusty):

return ( if ("supertabular" isInfixOf (writerTemplate opts)) then ( "\begin{supertabular}[c]" <> braces ("@{}" <> colDescriptors <> "@{}") $$ capt $$ headers $$ vcat rows' $$ "\end{supertabular}" ) else -- normal old behavior ( "\begin{longtable}[c]" <> braces ("@{}" <> colDescriptors <> "@{}") -- the @{} removes extra space at beginning and end $$ capt $$ "\toprule" $$ headers $$ endhead $$ vcat rows' $$ "\bottomrule" $$ "\end{longtable}" ) )

— Reply to this email directly or [1]view it on GitHub.

References

1. https://github.com/jgm/pandoc/issues/1023#issuecomment-132087361

[nvasilakis]

Changing the lines you suggested to:

\let\longtable\supertabular
\let\endlongtable\endsupertabular

while adding:

\usepackage{supertabular}
\usepackage{booktabs}

..still doesn't work:

! Undefined control sequence.
<recently read> \endhead

My goal is to have tables in two-column latex documents. Do you use a different way to achieve a similar result in your documents?

Regarding the psudo-patch I proposed, I know it's not optimal, it's just to get people going until a proper solution gets in. Another simple idea might be to just default to supertabular or xtab.

[jgm]

+++ Nikos Vasilakis [Aug 18 15 10:45 ]:

My goal is to have tables in two-column latex documents. Do you use a different way to achieve a similar result in your documents?

I've never tried this.

Regarding the psudo-patch I proposed, I know it's not optimal, it's just to get people going until a proper solution gets in. Another simple idea might be to just default to supertabular or xtab.

The problem is that if we do support it in some versions, people will have documents that depend on this behavior, and we'll have to keep it or make breaking changes.

One possibility would be to check for twocolumn in the classoption variable, and use a regular table environment if so. But, as I recall, there are still some issues relating to different handling of footnotes in table and longtable -- maybe other things too.

What are the advantages of supertabular or xtab over table?

[iamaziz]

As a workaround, someone (thanks!) has created a filter for using tabular instead of longtable in twocolumn environment (e.g. IEEEtrans, after some struggle I finally find a solution!! I tried it and it works), check it out here:

https://groups.google.com/forum/#!msg/pandoc-discuss/RUC-tuu_qf0/h-H3RRVt1coJ

To use it:

• copy the code from that link into some file table-filter.py
• Install pandocfilters
• use the filter when you convert you doc to latex,

e.g. $ pandoc SORUCE -o OUT.tex --filter table-filter.py

[jgm]

I tried this, and there are a number of issues with this filter, though it may be fine for simple cases:

1. The filter crashes if you have a table without headers.

2. Because the filter removes Table elements from the AST, the latex writer won't include

   \usepackage{booktabs}

in the header; you'll need to add this yourself through -H or a custom template, or by doing -V tables.

1. Relative column widths are not kept, so tables with long cell contents will stretch beyond page width.
2. Finally, footnotes don't work in these tables.

I'd be open to adding an option to pandoc to use regular tabular, if I could figure out how to make notes work properly, etc.

+++ Aziz Alto [Nov 29 15 02:52 ]:

As a workaround, someone (thanks!) has created a filter for using tabular instead of longtable in twocolumn environment (e.g. IEEEtrans, after some struggle I finally find a solution!! I tried it and it works), check it out here:

[1]https://groups.google.com/forum/#!msg/pandoc-discuss/RUC-tuu_qf0/h-H 3RRVt1coJ

To use it: * copy the code from that link into some file table-filter.py * Install [2]pandocfilters * use the filter when you [3]convert you doc to latex,

e.g. $ pandoc SORUCE -o OUT.tex --filter table-filter.py

— Reply to this email directly or [4]view it on GitHub.

References

1. https://groups.google.com/forum/#!msg/pandoc-discuss/RUC-tuu_qf0/h-H3RRVt1coJ
2. https://github.com/jgm/pandocfilters#installing
3. https://github.com/jgm/pandocfilters#what-are-pandoc-filters
4. https://github.com/jgm/pandoc/issues/1023#issuecomment-160400040

[iamaziz]

That's right! however in my case the filter does work with IEEEtran class without any significant issue. I do use many simple tables (with headers) in my docs, and it's not feasible to convert them manually.

I add \usepackage{booktabs} to the header of the template I use. Also another option (instead of using booktabs) would be to replace \toprule, \bottomrule, and \midrule with \hline in the filter.py.

The tables with long cells work fine in the IEEEtran, but not in the other twocolumn document classes (e.g. IEEEconf, [twocolumn]{journal} ).

I have not tried the footnotes. Thank you.

[basus]

I would really like to see this get resolved without resorting to running external scripts and such. Since two-column formats are pretty common for LaTeX documents would it be possible to have a separate output mode for them as part of pandoc itself?

[jgm]

I'd like to solve this too. What I need to know is how to reproduce our current table features using regular tabular. As noted above, two features that are a bit tricky are (a) footnotes in table cells and (b) relative widths in columns.

Once I've got this down, we could do several things. One option would be to have a command-line option to use regular tabular instead of longtable. Another would be to have tables inside a specially marked div treated specially (perhaps the special behavior should be longtable instead of regular tabular).

[basus]

I'd be willing to take a crack at this, possibly over the break. It looks like the relevant parts in the LaTeX writer are the functions blockToLatex (the Table case), tableRowToLatex and tableCellToLatex. I haven't used footnotes, either in markdown or pandoc, so I'll have to look into that.

[jgm]

I'd have no trouble doing the Haskell bits. I just need to know exactly what output to produce.

+++ Shrutarshi Basu [Dec 20 15 13:21 ]:

I'd be willing to take a crack at this, possibly over the break. It looks like the relevant parts in the LaTeX writer are the functions blockToLatex (the Table case), tableRowToLatex and tableCellToLatex. I haven't used footnotes, either in markdown or pandoc, so I'll have to look into that.

— Reply to this email directly or [1]view it on GitHub.

References

1. https://github.com/jgm/pandoc/issues/1023#issuecomment-166155761

[01mf02]

Hi, I have converted a multiline table example from the Pandoc user's guide to use tabular / table, and footnotes (courtesy of http://tex.stackexchange.com/a/35328) as well as relative widths seem to work.

The LaTeX code:

\documentclass{article}
\usepackage{hyperref}
\usepackage{tablefootnote}
\usepackage{booktabs}

\begin{document}
\begin{table}
\caption{Here's a multiline table without headers.}
\begin{tabular}{ c l r l }
\tabularnewline
\toprule
\toprule
\begin{minipage}[t]{0.15\columnwidth}\centering\strut
First
\strut\end{minipage} &
\begin{minipage}[t]{0.10\columnwidth}\raggedright\strut
row
\strut\end{minipage} &
\begin{minipage}[t]{0.20\columnwidth}\raggedleft\strut
12.0
\strut\end{minipage} &
\begin{minipage}[t]{0.31\columnwidth}\raggedright\strut
Example of a row that spans multiple lines.\tablefootnote{Footnotes also work.}
\strut\end{minipage}\tabularnewline
\begin{minipage}[t]{0.15\columnwidth}\centering\strut
Second
\strut\end{minipage} &
\begin{minipage}[t]{0.10\columnwidth}\raggedright\strut
row
\strut\end{minipage} &
\begin{minipage}[t]{0.20\columnwidth}\raggedleft\strut
5.0
\strut\end{minipage} &
\begin{minipage}[t]{0.31\columnwidth}\raggedright\strut
Here's another one. \tablefootnote{Another footnote.}Note the blank line between rows.
\strut\end{minipage}\tabularnewline
\bottomrule
\end{tabular}
\end{table}

\end{document}

[ickc]

Hi, will there be a command option/metadata variable for us to choose which kind of table will be used in LaTeX? From another thread it seems pandoc had changed the package it use for table but the old way wasn't kept. It would be nice if pandoc allow a choice of which conversion to use.

And is the use of table package open for discussion? I know that MultiMarkdown use tabulary, and probably tabularx has better footnote in table support (I once need to manually turn the tabulary table in mmd to a tabularx one). And I heard that tabu is supposed to be the newest and best package for table but haven't tried it yet.

[jgm]

See #2384, #1023, and related discussions in pandoc-discuss. I'd like to have a better solution here.

+++ ickc [Apr 26 16 22:56 ]:

Hi, will there be a command option/metadata variable for us to choose which kind of table will be used in LaTeX? From another thread it seems pandoc had changed the package it use for table but the old way wasn't kept. It would be nice if pandoc allow a choice of which conversion to use.

And is the use of table package open for discussion? I know that MultiMarkdown use tabulary, and probably tabularx has better footnote in table support (I once need to manually turn the tabulary table in mmd to a tabularx one). And I heard that tabu is supposed to be the newest and best package for table but haven't tried it yet.

— You are receiving this because you commented. Reply to this email directly or [1]view it on GitHub

References

1. https://github.com/jgm/pandoc/issues/1023#issuecomment-214978814

[01mf02]

@jgm, what do you think about my solution? I think it should be fairly easy to adapt the existing table export code in pandoc to the format I proposed, and I do not see any problems with it.

[jgm]

I've long had the goal of trying to stick to packages in the texlive-latex-recommended subset on debian. (I'd rather that users not have to download gigabytes of LaTeX just to use regular tables.) Unfortunately, the tablefootnote package is in texlive-latex-extra -- so that makes me reluctant to use this approach.

In the end, it might still be worth considering. Maybe we could have use of longtable or this approach be configurable.

+++ Michael Färber [Apr 14 16 10:00 ]:

Hi, I have converted a multiline table example from the Pandoc user's guide to use tabular / table, and footnotes (courtesy of [1]http://tex.stackexchange.com/a/35328) as well as relative widths seem to work.

The LaTeX code:

\documentclass{article} \usepackage{hyperref} \usepackage{tablefootnote} \usepackage{booktabs}

\begin{document} \begin{table} \caption{Here's a multiline table without headers.} \begin{tabular}{ c l r l } \tabularnewline \toprule \toprule \begin{minipage}[t]{0.15\columnwidth}\centering\strut First \strut\end{minipage} & \begin{minipage}[t]{0.10\columnwidth}\raggedright\strut row \strut\end{minipage} & \begin{minipage}[t]{0.20\columnwidth}\raggedleft\strut 12.0 \strut\end{minipage} & \begin{minipage}[t]{0.31\columnwidth}\raggedright\strut Example of a row that spans multiple lines.\tablefootnote{Footnotes also work.} \strut\end{minipage}\tabularnewline \begin{minipage}[t]{0.15\columnwidth}\centering\strut Second \strut\end{minipage} & \begin{minipage}[t]{0.10\columnwidth}\raggedright\strut row \strut\end{minipage} & \begin{minipage}[t]{0.20\columnwidth}\raggedleft\strut 5.0 \strut\end{minipage} & \begin{minipage}[t]{0.31\columnwidth}\raggedright\strut Here's another one. \tablefootnote{Another footnote.}Note the blank line between rows. \strut\end{minipage}\tabularnewline \bottomrule \end{tabular} \end{table}

\end{document}

— You are receiving this because you commented. Reply to this email directly or [2]view it on GitHub

References

1. http://tex.stackexchange.com/a/35328
2. https://github.com/jgm/pandoc/issues/1023#issuecomment-210048903

[01mf02]

I understand. How about then leaving longtable as the default and creating a command-line option to use tablefootnote? This would not create a dependency for users that "just want some table", and if they need non-breaking tables desperately enough to use a new command-line option, they are probably also willing to depend on some extra package. :)

(At least I am desperate enough. The table situation is by far my greatest Pandoc problem by far.)

[azrdev]

Assuming you're bound to longtable (e.g. by the IEEEtran documentclass), but want to use pandocs' tables, there is a workaround forcing the table into a single column here: http://tex.stackexchange.com/a/224096/90603. It breaks when the content becomes too much (starts to write over the other column), but at least you don't have to convert the table manually to TeX.

That said, a solution to this issue would be great! In the meantime, already a possibility to output a markdown table as simple tabular code would really ease the manual workaround.

[AndrewC-B]

I've spent quite a bit of time trying to work around this issue. While the workarounds which can be found in this thread (and in other places around the internet) work in some cases they fail in others (including my own relatively simple case*). As of the time of writing my determination is that if you want to produce simple tables for PDF output with Pandoc in a two-column layout the best option is to write raw Latex tables into your Markdown source.

From my point of view this isn't a great outcome.** We're also coming up for four years since this issue was opened, so I had hoped for some progress by now. Even just a cleaner workaround, as others have suggested, would be awesome.

Is this assessment correct? I'm not familiar with the inner workings of either Pandoc or Latex, so it's entirely possible I've missed something. It's also possible that something has changed in the past year or so since @jgm last chimed in, in which case it would be good to update this thread with the latest information (new workarounds, new official Pandoc-supported options, official decisions regarding this issue etc.)

---

* If anyone's interested, here's what I'm trying to achieve:

• A two-column layout document (IEEE).
• A table written in Pandoc's table syntax, with a caption (as per Pandoc's user guide).
• PDF output from the Markdown source files.
• In the output, the table should be rendered in the same place relative to the surrounding text as in the source file (ie: not Latex-floated off somewhere else).

---

** This doesn't mean I'm not grateful for all the work that's been done on this fantastic project!

[jgm]

You're right that we still use longtable in pandoc; I just haven't gotten around to exploring workarounds for two columns, but I'll try to prioritize this more.

The dev version now has a groff ms writer; you can do

pandoc -f ms -o output.pdf input.md

This can give you two-column output with a small tweak in the default template. It handles both math and tables (which I assume will work in two-column mode). Output is not quite as nice as PDF, though, and there are some other limitations. You might try it.