ct:capture_start/get interferes with messages to process under test

[TD5]

Describe the bug https://erlang.org/doc/man/ct.html#capture_get-1 seems to work by redirecting writes to stdout to be messages to the current pid. Those messages get queued up, with the intent to read them back later to get stdout. Unfortunately, this doesn't work when the test case itself is handling messages.

Here's an example of one situation where this crops up:

At various points, Dialyzer dequeues messages it receives from its worker child processes, sometimes throwing them away knowing it doesn't need certain results from them.

Sadly, this interacts badly with ct:capture_start/0 / ct:capture_get/1 and Dialyzer ends up dequeuing (and hence, deleting) some of what was written to stdout. The solution was to run Dialyzer in a sub-process, so the messages containing stdout go to the parent process's mailbox and Dialyzer is free to do what it wants with its own messages.

Fundamentally this seems to be an issue with the capture mechanism, since it seems to presume it's fine to send messages to the test case's pid and they'll never be consumed by the code under test!

Here was my workaround for capturing Dialyzer's stdout in a test:

run_dialyzer_capture(Analysis, Files, Opts) ->
    ct:capture_start(),
    TestCasePid = self(),
    RunDialyzer = fun() ->
        Ret = run_dialyzer(Analysis, Files, Opts),
        TestCasePid ! {dialyzer_done, Ret}
    end,
    _DialyzerPid = spawn_link(RunDialyzer),
    DialyzerRet =
        receive
            {dialyzer_done, Ret} -> Ret
        end,
    ct:capture_stop(),
    Stdout = ct:capture_get([]),
    {DialyzerRet, Stdout}.

In this case, the code that writes to stdout and receives messages is run as a new process, so that the test case which captures the stdout into its mailbox is not also the process which is reading from its mailbox. I suspect in general, you'd want to run the code-under-test in a sub-process to make sure its safe to redirect stdout to the current process's message queue.

To Reproduce Use ct:capture_start/0 in a test case, and within that test case, write to stdout, then consume the message from the mailbox. It will now no longer appear in the captured output from ct:capture_get/1.

Expected behavior I'd expect ct:capture_get/1 to work even if the test reads messages. The code in the description offers a way to do this by creating a new process.

Affected versions All recent versions I have used.

[TD5]

I am definitely quite new to this mechanism, so it's very possible that I am misusing the functionality somehow. In that case, maybe we can tweak the docs to make the intended usage clearer?

[IngelaAndin]

I actually have not used this myself but I discussed it with some coworkers and we think that what you say makes sense, and we see no reason to not change so it works like that. I am not sure when it can be prioritized, a PR would be welcome.