rewrite: feature request, per-file filter

[pluskid]

Hi, thanks for making the rewrite branch. It looks very promising. However, I would like to request for a per-file control. e.g. maybe a per-file directive like @file{Foo/bar.jl}. Here are some arguments about this.

• It is common practice to organize a module into several different files. For example, the demo for Lexicon.jl setup in Sims.jl handcrafted a way to dump per-file documents.
• The Julia module tends to be a flat one containing a lot of stuff (e.g. the Base module of Julia itself). A typical example, is that a module contains multiple types. Typically, I organize those types into different files. On the other hand, since Julia is using multiple-dispatch instead of OOP, so all the functions relevant to a type is not automatically grouped together. Displaying all the functions within a module that contains multiple types in alphabet order tends to be not very useful -- it might serve as a reference when the user knows exactly what he is looking for, otherwise the organization might be completely confusing.

If it is difficult to implement per-file filter. I think an alternative, or maybe even better way is to allow the dumped document to be of exactly the same order as it appeared in the source file. So we can add section titles, descriptions, and the documents should be organized nicely because the original source code should already been organized in a coherent way. It is like the whole source tree is a big IJulia Notebook, with description and code interleaved.

[pluskid]

After looking at the examples more closely, it seems rewrite already have per-function control, which looks very cool.

[MichaelHatherly]

Sounds reasonable to me.

All docstrings should contain a .meta[:path] value that tracks which file a docstring is from. With that writing an @file{...} directive should be doable with a little bit of work.

Handling ordering within a single file might be more of a problem since the line numbers aren't being tracked by the docsystem. We could possibly just search through the file and work out the order manually, but that would probably be quite brittle.

[pluskid]

@MichaelHatherly Thanks! That would be really great! :+1:

[MichaelHatherly]

@file{Foo/bar.jl} would be ambiguous since it's not a full path, but I wouldn't want to have to write out the full path every time... Possibly the path should be relative to the current module or file. Might also be worth supporting regex for paths to select multiple files at once.

Also a @moduledocs{MyModule} to splice in every doc from a module might be useful.

[pluskid]

Yes, assuming we are only talking about docs for a single package, path ambiguity might be less a problem. But regex is definite a great plus!

[MichaelHatherly]

assuming we are only talking about docs for a single package

Given how makedocs works you can merge any number of packages into a single doc, but it is probably safe to assume that's not going to see significant use over just documenting one package at a time.

For the actual syntax for this I'm thinking probably require passing a string or regex rather than a bare expression as in @file{file/path.jl}, ie. @file{"full/path.jl"} and @file{r"path.jl"} instead, to make it slightly easier to parse correctly.

Passing an ordering function to @file could potentially be useful as well:

@file{"path.jl", custom_order} or @file{"path.jl", (a, b) -> ...}.

[pluskid]

Yes, that sounds better. But I'm also a bit confused as I think those directives are written in markdown (templates). I thought they are just a customized DSL. Do you mean they can actually be arbitrary Julia expressions?

[MichaelHatherly]

Yes, they're embedded in markdown and initially are just parsed as strings. After that you can do just about anything with the string though, see the builtin ones for example https://github.com/MichaelHatherly/Docile.jl/blob/f758f8bb053085fa35b63748595a74de2f102ad0/src/DocTree.jl#L258.

For this particular case with @file using a string literal just helps match up with allowing regex syntax as well. If we were to leave out regex then I'd be fine with having it as @file{file/path.jl} instead.

[pluskid]

I see. Thanks for clarification. I mean I do not have any complain on what exactly the final syntax would look like. :smile: A consistent one must be better!