DocTestSetup shouldn't be re-executed with follow-up named doctests
In LLVM.jl, I'm using DocTestSetup to allocate an LLVM context which contains all derived LLVM resources. I'm also cleaning up any previous context, because Documenter.jl currently doesn't have a DocTestTeardown or something like it (https://github.com/JuliaDocs/Documenter.jl/issues/2566):
```@meta
DocTestSetup = quote
using LLVM
# XXX; clean-up previous contexts
while context(; throw_error=false) !== nothing
dispose(context())
end
ctx = Context()
end
```
At the same time, I was using named doctests in order to share variables between different doctests:
```jldoctest mod
julia> mod = LLVM.Module("SomeModule");
```
```jldoctest mod
julia> println(mod)
LLVM.Module("SomeModule")
```
This segfaults, because the DocTestSetup triggers again for the second block, destroying the context in which mod was allocated. This is surprising to me, but I realize it can probably not be changed without breaking existing doctests...
Contrived MWE demonstrating the issue:
```@meta
DocTestSetup = quote
# remove any previous buffer (JuliaDocs/Documenter.jl#2566)
try
ccall(:free, Cvoid, (Ptr{Cvoid},), task_local_storage(:buf))
catch
end
# allocate a new buffer
buf = ccall(:malloc, Ptr{Int}, (Csize_t,), 1)
ccall(:memset, Cvoid, (Ptr{Int}, Int, Csize_t), buf, 0, 1)
task_local_storage(:buf, buf)
end
```
```jldoctest ptr
julia> ptr = buf;
julia> unsafe_load(ptr)
0
```
```jldoctest ptr
julia> unsafe_load(ptr)
0
```
On my system, the second doctest doesn't cause a crash, but doesn't show the correct value.
I agree that it makes sense that the global setup should only apply once on named doctests. But my concern is that changing the behavior would be a breaking change. I guess we should just add an option for makedocs (and maybe @meta) where you override the behavior, so people can opt-in to the new behavior.
And yea, as noted in https://github.com/JuliaDocs/Documenter.jl/pull/2577#issuecomment-2724045257, I think the teardown should always be consistent with setup (so either for each doctest, or once per named block).