Splice a code snippet from a text/code file

[jebej]

It might be useful to have the option to include a .jl file in a page and display the code. This would make it easy to have examples in an example folder, but still be able to display them in the documentation. There might already be a way to do that in html however, but I do not know how.

This is just a nice to have, as it is possible to do this simply by copying the code over and keeping it synced manually.

[mortenpi]

Nope, not currently possible, but would be something that could be done if someone needs it. Sounds like a new ```@-block (@include?). But I would leave the implementation as up for grabs though.

[s-celles]

Hello,

I was also looking for a similar feature to include sample code to documentation (but keeping it as a .jl file) I asked recently on Gitter without any positive result. You might have a look at this Discourse discussion https://discourse.julialang.org/t/best-practices-for-examples-and-ci/7304

Kind regards

[KristofferC]

The way you do it right now is:

````@eval
Markdown.parse("""
```
$(readstring("PATH_TO_FILE"))
```
""")
````

[s-celles]

Does it enable syntax color highlighting for Julia ?

[tpapp]

I am trying to implement this, but need a bit of help understanding the code.

1. Is it correct that the only thing I have to do is write an expander, say for @splicecode (bikeshedding welcome 😄), in src/Expanders.jl? For this I need the the abstract types, the Selector.order, Selector.matcher, and Selector.runner.

2. What should it do? Ie if I have the code snippet read (as as string code) and the language parsed (lang, defaults to "julia"), what should the Selectors.runner insert? OK if I make a

page.mapping[x] = Markdown.MD("```$lang\n$code```\n)

or is it something more complex?

Sorry if the questions are trivial, this is my first time looking at the code of Documenter.jl.

[mortenpi]

Does it enable syntax color highlighting for Julia ?

You can always add the language to the code block, e.g.:

````@eval
Markdown.parse("""
```julia
$(readstring("PATH_TO_FILE"))
```
""")
````

@tpapp: Awesome! Yes, an expander is what is needed.

The other expanders often construct specialized *Node objects, in which case the rendering code can decide to do something more sophisticated with it. But just splicing in an ordinary code block is probably enough here.

The mapped object should be a Markdown.Code(lang, code) though.

For the interface, I would personally probably lean towards something similar to @meta. I.e. the user gives the file name with key/value pairs. This way it can be expanded in the future, if need be. In the following example, File would be mandatory, Language optional (would allow the user to specify the language explicitly; the default should either be Julia or determined from the extension).

```@splicecode
File = "..."`
Language = "julia"
```

A bit of thought is needed for how the file path will be resolved. Should we assume that the spliced files are always external to the src/ directory? In that case, whatever we go with, it should be consistent with #552, I think. What we settled on there is to have a new makedocs option called external_root.

[tpapp]

@mortenpi: what I was thinking of is a syntax

```@splicecode
file = "..."                   # mandatory
lang = "julia"                 # optional
lines = 1:end                  # optional, intepreted within the range selected below
after = "# CODE BLOCK 1 START" # optional, mark the beginning of a code block
before = "# CODE BLOCK 1 END"  # optional, mark the end of a code block
```

so that the user could select based on lines and also strings marking some code snippet.

My understanding is that I can use Utilities.parseblock to get these and eval the right hand sides, is this correct? Is there any validation mechanism, so that

file = run(`rm -Rf ~/`)

is excluded? Or is it just plain eval?

What I am unclear about is external, should

file = "../../src/include_this.jl"

and similar not take care of this? How can I use external in this context?

[mortenpi]

LGTM @tpapp! I would stick to CamelCase with the keys though, to be consistent with the other at-blocks.

parseblock just parse()s into AST and does not eval, as far as I can tell.

Regarding external.. yes, you are right I think.. by default the path should be considered relative to the current .md file (or .jl file containing the docstring with the @splicecode block). File = external("...") could be something useful for the user though, so you could have File = external("examples/sth.jl"), instead of File = "../../examples/sth.jl".

[cossio]

I was looking for this functionality. Was there a PR made in the end?

[tpapp]

@cossio: I didn't follow up on this. FWIW, I don't think that source code should be lifted into documentation like this in most cases. For literate programming, we have Literate.jl.

[cossio]

Thanks. Where can I look at some examples of Literate + Documenter interaction? Or Weave + Documenter?

[tpapp]

https://fredrikekre.github.io/Literate.jl/v2/documenter/

[cossio]

Oh yes I had read that, thanks. But what I meant was how to generate the pages automatically during CI. Like, what should I put in the make.jl file of Documenter to make it generate also the Literate pages and have them appear linked in the documentation site? Probably there are some packages out there doing this already, I would just like to inspect their source code to see how they do it.

[odow]

I think this issue could be closed. There is very little reason to include code snippets from source (no people asking for 5 years), and we have Literate.jl which is serves the purpose of embedding examples etc.

There's an open issue for explaining how to use Literate: https://github.com/JuliaDocs/Documenter.jl/issues/2200

[mortenpi]

Actually, let me reopen this: let's refocus this issue around the splicing feature: https://github.com/JuliaDocs/Documenter.jl/issues/499#issuecomment-346994594

I actually do have the need for something like that. Although, I think it should be a plugin package in first instance.