`@doc` changes behaviour during testing

[lgoettgens]

We have some code doing automatic creation of docstrings in AbstractAlgebra.jl including some test around that. These tests started to fail a few days ago on julia nightly (ref https://github.com/Nemocas/AbstractAlgebra.jl/issues/1608), due to the @doc macro no longer returning the docstring but only nothing.

MWE:

==> ./Project.toml <==
name = "MWE"
uuid = "f8fce2d4-4067-48d1-bd5a-dca4ed90cd6a"

[extras]
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[targets]
test = ["Test"]

==> ./src/MWE.jl <==
module MWE

"""
    foo()
"""
function foo()
    return 42
end

end # module MWE

==> ./test/runtests.jl <==
using MWE
using Test

@test contains(string(@doc MWE.foo), "foo()")

When running the tests from the REPL using include("test/runtests.jl"), everything is fine. However, running them via ] test fails.

A bisection has shown that the regression was introduced in https://github.com/JuliaLang/julia/commit/c1a7bb2787695334c56f504d1f8447dadeb29e3a (https://github.com/JuliaLang/julia/pull/53220), so this could as well be a Pkg.jl issue.

[vtjnash]

Yes, this was a REPL feature that was (apparently) not supposed to be accessed by code. Typical doc usage would have see-also links rather than literally copy text. You might need to load REPL currently in your tests to get them working again

[KristofferC]

I'll put this on the milestone. From the docstring:

The @doc macro can be used directly to both set and retrieve documentation / metadata.

Nothing says this is a REPL feature and it doesn't really make sense for it to be.

[Tortar]

Dup of #52986