Version doctests

[bramtayl]

Is there a mechanism to make doctests conditional on Julia version? Printing seems pretty unstable version to version

[fredrikekre]

Just set the doctest kwarg to be conditional?

makedocs(
    doctest = VERSION == v"1.0.0"
   ...
)

or w/e.

[bramtayl]

I was hoping for something like:

doctest VERSION == this_version this printing

doctest VERSION == another_version another printing

[mortenpi]

So you mean an argument to an jldoctest block? Technically it would be doable, but I am a bit sceptical whether this is a good idea. E.g. in docstrings, you'd then end up with multiple sets of doctests that all show up in the REPL.

I do think that it would be better to just target a single Julia version with the doctests, as Fredrik suggested. It's not even a problem to target the latest Julia -- updating all the doctests with the :fix option every few months is relatively painless.

[bramtayl]

I'm mostly interested in this because I want to maintain support for both 1.0 and 1.1, and I run doctests as part of CI. Preferable the jldoctest block would only run and render on its matching julia version

[mortenpi]

I'm mostly interested in this because I want to maintain support for both 1.0 and 1.1, and I run doctests as part of CI.

I would version guard that, so that you doctest only when the Julia version matches. I don't think you really gain much from testing the doctests with multiple Julia versions. If they work for one Julia version, everything is probably fine for all versions (I assume you still have normal tests that verify the APIs on all versions). Do you have a particular example where it would be likely that this assumption does not hold?

[bramtayl]

I assume you still have normal tests that verify the APIs on all versions

Not usually. Or to put it another way, it's rare for me to write a test and then decide it wouldn't be useful for the user to see it. E.g. LightQuery.jl has an extensive tutorial which doubles as a doctest, as well as regular doc-string doctests, but no regular tests.

[bramtayl]

Maybe a bit more generally, adding a condition under which doctests will run and display:

doctest condition = x -> VERSION == v"1.0"

[bramtayl]

Eh, in newer packages, I've decided to only run doctests on latest julia, and replicate them all as regular tests. It's a little bit less elegant but more robust since printing changes so much between version. Maybe it would be nice to have a note in the docs about doctests being fragile.