Custom styles for ODT (and maybe LaTeX and RST) writers

[matthijskooijman]

Currently, you can customize formatting by providing some customized styles in a reference odt/docx file to pandoc. These styles are copied into the generated document and applied to selected elements in the document.

However, this only supports a limited, hard-coded list of styles. E.g. the 'Heading 1' style is applied to level-1 headings, the 'Text body' style to regular paragraphs, the 'Bullet Symbols' style to bullets. On top of this, I'd like to be able to specify other styles to use, on a block-by-block basis.

My usecase is that I'm writing a book, and my publisher requires the text to be delivered in docx or odt format, using a set of custom styles with non-standard style names. I have two challenges:

1. Regular styles have different names, so I need to use e.g. "Publisher Heading 1" for first-level headings.
2. Some parts of my document must use completely different styles, which have no equivalent in the pandoc AST (e.g. notes for the layouter must use the "Layout Notes" style).

If I have some way to specify custom style names in the AST, I can use a pandoc filter to introduce the proper style names in the AST and have the writers use the right styles. Ideally, I'd be able to specify the style names for challenge 2 above directly in my markdown source.

Attributes seem like a good fit for this, so that would work out to something like this in markdown:

# My Header {style="Fancy Header 1"}

Note that this is already supported by the current markdown parser, and results in this JSON:

[{"unMeta":{}},
 [{"t":"Header","c":[1,["my-header",[],[["style","Fancy Header 1"]]],
 [{"t":"Str","c":"My"},{"t":"Space","c":[]},{"t":"Str","c":"Header"}]]}]
]

However, converting this to ODT or DOCX drops the attributes completely.

I'm aware that not all elements currently allow specifying styles, but that's another discussion. What I'd like to see is that, for all elements that currently support attributes, the "style" attribute be interpreted as a style name in the ODT and DOCX writes.

ODT and DOCX have a distinction between paragraph and character styles, but it seems that using the paragraph styles for block elements and character styles for inline elements is sufficient?

Does this sound reasonable? Is the "style" attribute the right fit here? For the HTML writer, the style attribute should contain a bit of raw CSS. I don't think we can have a single syntax that works for both HTML and ODT/DOCX, though perhaps we can have something that allows specifying two distinct attributes to allow working with both? At first glance, the class is more appropriate than an attribute, since a class is also a name that indirectly specifies formatting to be used. However, a class can be specified multiple times, which I don't think is applicable to DOCX / ODT styles? Or perhaps just use the first / last class name specified?

I'll likely spend a bit of time implementing this for my own needs, but I'd rather know up-front if this has any chance of being merged.

@mpickering, It seems that such a style= attribute could perhaps also be used by an asciidoc reader to fix #1234 / #1235.

[gbjbaanb]

It is possible to alter the styles in the docx after the generation is complete. For example, here I set the style for all tables to BlueTableStyle (which is a style in the template dotx I use to generate docx from pandoc). The styles are contained in the document.xml file inside the docx archive.

Obviously this is not an ideal solution and being able for pandoc to spit out the desired style name given the right directives would be great, similar to how you can specify them for HTML. I don't see why they would need different input syntax, the use-case for generating different styles based on output type is a can of worms best left closed.

7z x -y %~1.docx word\document.xml 
sed "s/<w:tblStyle w:val=\"TableNormal\"/<w:tblStyle w:val=\"BlueTableStyle\"/g" word\document.xml > word\document2.xml
copy word\document2.xml word\document.xml /y
7z u -y %~1.docx word\document.xml

[krtek4]

I am in the exact same situation : writing a book and the publisher insists on docx file format using their custom styles.

Is there any chance this could get implemented ?

The workaround proposed works really great when you want to style all given elements the same way, or rename a given style to some other one. But when just some elements needs to use a specific style I don't see how it can be made to work.

Thanks !

[matthijskooijman]

@krtek4, I ended up using docutils's rst2odt tool with the --odf-config-file and --stylesheet options to customize the output styles, which ended up suiting by needs just enough (with a few minor customizations to better handle images, code, etc.). I haven't found time to clean up my build process well enough to publish it, but if you're interested I can do a quick sweep, tar it up and send it to you. If so, drop me an email to not further pollute this issue.

[josdirksen]

I'm writing a book for Packt, which also uses a crappy docx template. I'm now using pandoc to convert my markdown file to docx, and afer that use a couple of XSLT steps to convert the styles from pandoc to the styles required by the template from Packt. This seems to work ok for the styles I'm currently using (basic bullets, lists, images etc.)

I haven't needed tables yet, but that should also be doable I guess. If you're interested let me know.

[krtek4]

Hi @josdirksen,

Finally I am use rst2odt like @matthijskooijman suggested. The tool allows for custom mapping from node types to ODT styles. For now I am able to manage with that quite well.

Hadn't thought of XSLT at the time, but that is also definitively a great idea :) Good continuation with your book.

[mb21]

Is this request fulfilled with http://pandoc.org/MANUAL.html#custom-styles-in-docx-output ? (at least for docx)

[krtek4]

As far as I can tell, this is exactly what is needed for docx. It is really similar to the feature I am using in rst2odt

[mb21]

Since this feature was implemented in docx, we might think about doing the same for ODT, ICML and maybe LaTeX and RST writers...

[matthijskooijman]

From a quick glance at the docs, I think what I requested in this issue would be made possible with that new feature. However, I've long since finished writing my book, so I have no specific interest in this feature anymore, so feel free to close this issue (or leave it open if someone else wants to do actual testing of the feature and report back).

[simongareste]

@mb21: I am very much interested in this for odt, and available to test.

[mb21]

@simongareste Personally, I'm not too familiar with the ODT format... feel free to take a look at the writer generating the OpenDocument format and the one packaging it into a .odt zip, pull requests welcome!

[marcban]

Unfortunately I'm not a Haskell developer, but here are some specs to help a little:

• source md file : this is a <span custom-style="mystyle">custom style span</span>
• in content.xml part of .odt file, this should generate:

<text:p text:style-name="Text_20_body">
  this is a <text:span text:style-name="mystyle">custom style span</text:span>
</text:p>`

• in styles.xml part, this should add a style (if not present) under <office:styles> tag, with value:

<style:style style:name="mystyle" style:family="text">
	<style:text-properties fo:font-weight="bold"/>
</style:style>

I propose that the custom style is bold by default, so that it will be visible in the output. Of course, by using a custom template, the user can define his own style.

[mb21]

I'm reposting here some discussion from the now closed https://github.com/jgm/pandoc/issues/2542

+++ Mauro Bieg [Dec 31 15 09:38 ]:

In LaTeX those divs/spans could be rendered as custom environments/commands. This was actually proposed by hadley in https://github.com/jgm/pandoc/issues/168#issuecomment-136480787, or do you think the concepts are not analogous enough?

+++ jgm:

I want to avoid generating environments and commands that aren't defined (and similarly for styles in Word and ICML). If we parse the styles, and thus know what is available, that may not be a big problem. In LaTeX it's harder, because commands and environments may be defined in included packages. The idea of having a special prefix like style- might be a good one.

Since the following already works for docx output:

::: {custom-style=poetry}
My example poem,
is bad.
:::

the idea is to change the syntax (and probably AST representation) to use a class with the style- prefix:

::: style-poetry
My example poem,
is bad.
:::

and make it work for ODT and ICML (and maybe even LaTeX and RST) as well (although for RST output, pandoc already uses the role attribute). For example LaTeX output:

\begin{poetry}
  My example poem, is bad.
\end{poetry}

[ghost]

Writing custom-style everywhere a custom style is required is redundant and differs in behaviour from HTML output. Consider:

::: {.projection}
    Executing reverse bias polarity on neurowafer 1 of 7.
    Estimated time remaining: 3h 39m 57s
:::

Thanks to #168, this produces the expected output:

<div class="projection">
<pre><code>Executing reverse bias polarity on neurowafer 1 of 7.
Estimated time remaining: 3h 39m 57s</code></pre>
</div>

However, when the output is ConTeXt, this produces the astonishing result:

\starttyping
Executing reverse bias polarity on neurowafer 1 of 7.
Estimated time remaining: 3h 39m 57s
\stoptyping

I consider it astonishing because the {.projection} class is swallowed, silently, instead of being retained in some manner, such as:

\starttyping[class=projection]
Executing reverse bias polarity on neurowafer 1 of 7.
Estimated time remaining: 3h 39m 57s
\stoptyping

Since pandoc cannot presume environments exist, it would be convenient if there was a command line argument that indicates that they do. Consider:

pandoc -t context --tex-environments --top-level-division=chapter file.md -o file.tex

Here, the --tex-environments option indicates that the developer (author) has created the required environments and that pandoc can use them. For ConTeXt, this could resemble:

\startprojection
\starttyping
Executing reverse bias polarity on neurowafer 1 of 7.
Estimated time remaining: 3h 39m 57s
\stoptyping
\stopprojection

Returning to the HTML output, if the source document contains:

::: {.style-projection}
    Executing reverse bias polarity on neurowafer 1 of 7.
    Estimated time remaining: 3h 39m 57s
:::

This would produce:

<div class="style-projection">
<pre><code>Executing reverse bias polarity on neurowafer 1 of 7.
Estimated time remaining: 3h 39m 57s</code></pre>
</div>

Clearly, the style- is superfluous and would only be necessary because there is no other way to coerce pandoc into generating LaTeX/ConTeXt environments. IMO, a command line argument would allow the Markdown document to be written as per the author's descriptive intent, without being encumbered with knowledge of the internal workings of pandoc (that is, how it treats Markdown differently depending on whether the output is HTML or TeX).

WDYT? @adityam? @jgm?

[jcbagneris]

Hello there,

I do agree that it would be convenient to have a way to tell pandoc "create an environment, I know what I am doing" but instead of a general option, I guess that an extension would be more adapted.

Something in the line of:

pandoc -f markdown -t latex+create_envs

That would allow for the extension to be output dependent, which is probably needed.

Fenced divs are great, let's make good use of those :)

Thanks

[ghost]

An extension would also work, though my preference would be for slightly different names:

pandoc -f markdown -t context+environments
pandoc -f markdown -t latex+environments

[jcbagneris]

Ah, I just meant to advocate for extension vs. option. I don't really care about the name, provided it's meaningful. +environments is ok for me.

[mb21]

In order to use the fenced div class syntax for custom-styles, the user has to specify his intent to use the div as such an environment - either by switching on an extension (e.g. +environment) or using a prefix like style- on the class.

The prefix is more verbose, but also more explicit: you then could have multiple divs in the same document and have only some converted to custom-styles aka environments. The writers could then strip the style- prefix when creating the style name. Do you think writing style- a few times is so bad?

[iandol]

On the duplicate #4139, I had asked for just a simple translation from ::: style ::: to a paragraph style in a format such as DOCX. The argument here is that to make this universal across other writers like LaTeX, an implicit conversion is not preferred by @jgm (although implicit conversion is the default for HTML presently). So the current options are to manually enable a universal mechanism (via either style- in the document or a +environment).

A third option is that this could depend on the writer, implicit for ODT/DOCX/HTML, explicit for LaTex and friends that change the actual output structure. I suspect this is also not preferred by @jgm, but Pandoc already has implicit HTML support and I think a div.class is semantically identical to a paragraph.style which is why the implicit rule could be extended to this type of document structured output...

[ghost]

Do you think writing style- a few times is so bad?

Yes. It's redundant, verbose, inconsistent, and assumes pandoc-specific implementation knowledge. (Aside, there's no way to know how many times a style will be used in a given document.)

The writers could then strip the style- prefix when creating the style name.

Seems to violate the KISS principle: it incurs extra work (post-processing) to strip the prefix. Simpler (for writers) to not use or require a prefix altogether.

[mb21]

@DaveJarvis but you agree that the toggle-by-extension-approach has the disadvantage that in one document, you cannot use some divs for custom-styling while using other divs for other purposes, right?

[ghost]

you cannot use some divs for custom-styling while using other divs for other purposes

There can be options for both scenarios. For example, the custom styles could be explicitly listed once on the command line (or read from an external data source), rather than repeated throughout the text, freeing the unlisted div demarcations for other purposes. Or, depending on what list is shorter, an option to list the unstyled divs may be useful.

Also, a style- prefix suggests that the class is used for style-ing, which hints at presentation logic. As you pointed out, classes can be used for more than styling. Whether and how content marked with classes is styled or otherwise manipulated belongs outside of the document.

$ pandoc --styled poetry,stanza,prose ...
$ pandoc --styled-file styled.txt ...
$ cat styled.txt
poetry
stanza
prose
$ pandoc --unstyled foo,bar ...
$ pandoc --unstyled-file unstyled.txt ...
$ cat unstyled.txt
foo
bar

[mb21]

So we have three options now (the exact naming/implementation is also still debatable):

• mark up divs with style- prefixed class
• +environments extension that would tell pandoc to treat all divs as custom-styled-environments
• --styled argument that lists all divs to be styled

I'd like to get some feedback from more people on which of the three they prefer... @jgm?

[jkr]

I think I prefer (a) explicit over implict, and (b) not having to change the command-line every time you change the source file (or consult a long file as reference for the sake of constructing a command line). For those reasons, FWIW, I guess I like the first option (style-) and pretty strongly dislike the others.

• I think +environments could have major side-effects in moving to word-processing docs, where you might not even know the names of all possible styles in your reference file.

• Keeping a list of all environments to be made into a custom style seems overly difficult to maintain. I can't think of another place in pandoc where the user has to specify content from within the document on the command line.

So -- style- seems the best of the three. Perhaps the least elegant, but more easier to use and maintain precisely for that reason.

BUT I'm not sure I see why style-* is preferable to custom-style=* (or some shorter key-value attr, style=*). It saves a few keystrokes, and replaces an = with a hyphen, at the expense of parsing values, and essentially creating an inconsistent ad-hoc syntax for certain class attributes. It's not a big deal, but it seems a bit more obscure for not that much gain.

[ghost]

How would I have to mark-up a paragraph so that the same class is generated in HTML and ConTeXt without having to change the source document?

I think +environments could have major side-effects in moving to word-processing docs, where you might not even know the names of all possible styles in your reference file.

Then don't use it when generating those formats?

Consider a source document that must be styled differently depending on the output format (HTML, TeX, ODS, etc.). Changing the source document (either manually or by post-processing) for different output document formats subverts some benefits of using Markdown (output format agnostic, clean separation of content and presentation). I think we're agreed on this point.

Ideally, the source Markdown document would not include pandoc-specific instructions (mark up) to control pandoc's output behaviour, either. This is where we disagree.

Perhaps different modes could allow both types of documents? One where the styles are embedded in the document, for those for whom maintaining the list would be burdensome, and another mode where the styles are listed for those who know the list of styles rarely changes? This gives people the freedom to choose how they want to mark up their documents.

[iandol]

I also feel it is against the spirit of Pandoc to have to change the source document for different output formats, so vote against style- or other fussy in-doc markup. I don't need to have different "purposes" for divs, I would only use divs / spans to add a structural separator (named style in a word processing doc or a style on a HTML element) to a section of a document; wouldn't this be the major use of this feature? +environments would the simpler option in my use case if we have to make this explicit in the pandoc settings. If we had to use --styled I'd argue the logic is inverted, we should use --unstyled, i.e. exclude divs that we don't want converted rather than those we do, but again I'm coming at this from a HTML/ODT/DOCX perspective where style is a semantic label for a container.

So this complexity only exists for LateX-family writers (where divs are used for multiple uses?), so I raise the simplest option of all again: why not make other formats like word processors implicit like HTML already is?

[mb21]

@DaveJarvis and @iandol, you are mentioning not having to change the source document for different output formats. While I would usually agree with you that the pandoc conversion should "just work", I can't really see this happening here.

If you want to take advantage of custom styles/environments defined in a reference.docx or custom LaTeX template, you'll always have to adjust the input to contain the names of those styles/environments. It is in the very nature of a "custom" environment. It's not a standardized element like bold or a link that pandoc can automatically translate. If we blindly take all the classes in HTML input and automatically produce docx styles with the same name, you'll have nothing gained, since those style-names in all likelihood don't exist in your reference.docx.

[iandol]

HTML is exactly the same, you can add a class to an element which may or may not be present in the CSS. The element has been semantically tagged, but unless the CSS does something presentational with it, it will not be visualised. There is no conceptual difference in my mind to what I would expect of DOCX (except the conversion may create default styles if none exist in the reference.docx, but they will inherit from Normal and thus not be visible, that is OK). I don't believe HTML is a special case conceptually, thus don't think it should be a special case practically…

The point is the semantic tags are now present in the document. In Word you simply edit the style of one of these tagged paragraphs, and all the others with the same semantic tag will adjust automatically, just as if we add a CSS rule to the HTML... The markdown author obviously wants a tag, as why would he use a fenced div with an explicit class name otherwise?

My knowledge of LaTeX is weak, so can't comment on how environments are conceptually considered there.

[mb21]

the conversion may create default styles if none exist in the reference.docx, but they will inherit from Normal

Will this not lead to a huge list of styles that clutter the GUI in Word?

In LaTeX, the styles cannot (easily) be generated but need to be defined in the template before you can use them (otherwise the PDF compilation will fail).

[iandol]

No, by default Word or LibreOffice does not show all styles in its UI, indeed I consider it a deficit as it encourages a mess of adhoc formatting and users ignorant to the benefit of styles... I always enable the optional style sidebar in both while I work. And both offer good tools to manage large style lists even if a user were to add many different named fenced divs!

More fundamentally, a user has deliberately used a fenced div and deliberately given it a class name. They are deliberately identifying this paragraph/span. It seems to me that the default behaviour should be, just as for more specific markup like code blocks etc. to convert this deliberate intent as the default. It aligns better with the existing HTML behaviour, the document structure, and the tools afforded by the editors for these output formats...