Documenter.jl icon indicating copy to clipboard operation
Documenter.jl copied to clipboard

DocTestSetup shouldn't be re-executed with follow-up named doctests

Open maleadt opened this issue 1 year ago • 1 comments

In LLVM.jl, I'm using DocTestSetup to allocate an LLVM context which contains all derived LLVM resources. I'm also cleaning up any previous context, because Documenter.jl currently doesn't have a DocTestTeardown or something like it (https://github.com/JuliaDocs/Documenter.jl/issues/2566):

```@meta
DocTestSetup = quote
    using LLVM

    # XXX; clean-up previous contexts
    while context(; throw_error=false) !== nothing
        dispose(context())
    end

    ctx = Context()
end
```

At the same time, I was using named doctests in order to share variables between different doctests:

```jldoctest mod
julia> mod = LLVM.Module("SomeModule");
```

```jldoctest mod
julia> println(mod)
LLVM.Module("SomeModule")
```

This segfaults, because the DocTestSetup triggers again for the second block, destroying the context in which mod was allocated. This is surprising to me, but I realize it can probably not be changed without breaking existing doctests...


Contrived MWE demonstrating the issue:

```@meta
DocTestSetup = quote
    # remove any previous buffer (JuliaDocs/Documenter.jl#2566)
    try
        ccall(:free, Cvoid, (Ptr{Cvoid},), task_local_storage(:buf))
    catch
    end

    # allocate a new buffer
    buf = ccall(:malloc, Ptr{Int}, (Csize_t,), 1)
    ccall(:memset, Cvoid, (Ptr{Int}, Int, Csize_t), buf, 0, 1)
    task_local_storage(:buf, buf)
end
```

```jldoctest ptr
julia> ptr = buf;

julia> unsafe_load(ptr)
0
```

```jldoctest ptr
julia> unsafe_load(ptr)
0
```

On my system, the second doctest doesn't cause a crash, but doesn't show the correct value.

maleadt avatar Aug 30 '24 12:08 maleadt

I agree that it makes sense that the global setup should only apply once on named doctests. But my concern is that changing the behavior would be a breaking change. I guess we should just add an option for makedocs (and maybe @meta) where you override the behavior, so people can opt-in to the new behavior.

And yea, as noted in https://github.com/JuliaDocs/Documenter.jl/pull/2577#issuecomment-2724045257, I think the teardown should always be consistent with setup (so either for each doctest, or once per named block).

mortenpi avatar Mar 16 '25 07:03 mortenpi