Support for Pandoc/Rmarkdown-style [@citations] possible?

[Anaphory]

Describe the problem/need and solution

Context I am trying to integrate a JOSS paper into my documentation. According to https://joss.readthedocs.io/en/latest/submitting.html#example-paper-and-bibliography citations are supposed to follow the Rmarkdown format:

For a quick reference, the following citation commands can be used:

• @author:2001 -> "Author et al. (2001)"
• [@author:2001] -> "(Author et al., 2001)"
• [@author1:2001; @author2:2001] -> "(Author1 et al., 2001; Author2 et al., 2002)"

Problem / Idea Myst supports references, but there seems to be no way for @handle-style citations.

Solution Is it difficult to add support for this kind of reference elements? Would it break any compatibility assumptions?

Benefit The Rmarkdown style of citations is not used only in JOSS, but I have seen it in other places, as well.

Guide for implementation

No response

Tasks and updates

No response

[welcome[bot]]

Thanks for opening your first issue here! Engagement like this is essential for open source projects! :hugs:
If you haven't done so already, check out EBP's Code of Conduct. Also, please try to follow the issue template as it helps other community members to contribute more effectively.
If your issue is a feature request, others may react to it, to raise its prominence (see Feature Voting).
Welcome to the EBP community! :tada:

[chrisjsewell]

Hey @Anaphory, so at the moment no it does not, but I was actually playing around with this recently in https://github.com/executablebooks/MyST-Parser/tree/pop-bibliography and https://github.com/chrisjsewell/sphinx-glossary

Currently, we rely on sphinxcontrib-bibtex for bibliographies; its great but, being an external extension, means you can't have this kind of integrated syntax.

Ideally I would like to have a solution with this kind of syntax, but also that was not just restricted to reading bibtex files; supporting more common formats like JSON/YAML/TOML. Also, not just restricted to bibliographies; conceptually it is just a reference to a key in an external file, and you can also use it for things like glossaries.

But certainly integration with JOSS would be nice

[chrisjsewell]

Out of interest, was there any other issues you had with JOSS integration?

[Anaphory]

I haven't tried much yet, I'm still drafting the paper. I'll look in more detail later and keep you updated if I encounter any other discrepancies between Sphinx's assumptions and JOSS's.

I don't know any of the internals of MyST and am not really a parser person, but if it turns out I can help with the implementation, I'll see what I can do.

[chrisjsewell]

That's great cheers, yeh no worries, primarily this just gives more prominence to my feeling that this would be nice to have in MyST

[Anaphory]

Actually, there is one: The paper assumes that all section headings are first-level headings. In my docs, I would of course want them to be h2 or lower. I assume I can tell Sphinx somewhere to shift all headings inside a specified MyST file down by N levels, but I haven't found one yet.

Looking for it, I found the MyST optional syntax page, which suggests that existing markdown-it-py plugins can be turned into MyST syntax extensions. There happen to be two (!) markdown-it plugins for promising support of Pandoc-style citations, so I guess that's a good starting point? Those two plugins are for JS markdown-it, mind you, not for markdown-it-py.

[chrisjsewell]

which suggests that existing markdown-it-py plugins can be turned into MyST syntax extensions

yep, the parsing of [@citations] syntax is relatively easy, its just then what you "do" with that; how do you link it up to a bibtex file, and get it to output nice HTML/LaTeX etc

The paper assumes that all section headings are first-level headings.

As in here, they would assume all H1?

# H1
## H1
### H1

or you want to be able to do:

# H2
## H3
### H4

[Anaphory]

I have and must have for the paper

# Summary

In my paper-inside-the-docs, I would want

<h2>Summary<a class="headerlink" href="#summary" title="Permalink to this headline">¶</a></h2>

with an <h1> that I set elsewhere, maybe in an rst file that provides a wrapper between paper manuscript as-is and documentation. If you have an idea, cool; if you think it's a different enhancement proposal, we should probably move it to a separate issue.

[Anaphory]

yep, the parsing of [@citations] syntax is relatively easy, its just then what you "do" with that; how do you link it up to a bibtex file, and get it to output nice HTML/LaTeX etc

Is there already a conceptual issue with this:

• So you have a data source (BibTeX, YAML, JSON) that maps keys to entries in some format-specific syntax, and you have some transformation (eg. CSL, a bibstyle, your idea of a glossary could eg. italicise the first occurence of a term and not change later ones) that translates entries into tokens, subject to further manipulation.

or does the issue ‘only’ start with which data source, how to parse it, which transformation, how to specify it, how to run it, and how the whole system interacts?

[chrisjsewell]

Yep exactly, the basic concept is fine, it's just handling all cases like:

• What if you have multiple sources
• where do you output the definitions
• handling referencing between the terms and their definitions
• formatting of the terms/definitions

[arwedus]

@chrisjsewell: Could myst simply support the rst feature "citations"? https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#citations It is basic but does what we currently need, i.e. named footnotes 🙂