Add note that using a handle to the Proc object inside a 'dies-ok{}' test will not capture an Exception by an external program

[tbrowder]

Problem or new feature

During recent development of a module I discovered, thanks to @ugexe's help, that running an external program using test 'dies-ok{}' will fail if one uses a handle to the Proc object from 'run'. (Note that seems to contradict the existing docs.)

Given a simple external program 'bad-prog.raku':

#!/usr/bin/env raku;
die "bad prog";

A good test file 'good-test.t':

use Test;
dies-ok {
    run "./bad-prog.raku";
}

Running it results in success:

raku good-test.t 
# OUTPUT: 
ok - 1

A bad test file 'bad-test.t' with the handle to the Proc object;

use Test;
dies-ok {
    my $p = run "./bad-prog.raku";
}

Running it results in a failure of the test:

raku bad-test.t 
# OUTPUT: 
not ok - 1
# Failed test at /bad-test.t line 2

[coke]

    run "-I", "./bad-prog.raku";

To be clear, this is trying to run a program named "-I" - it looks like you're expecting this to run "raku" ?

use Test;
dies-ok {
    my $p = run "-I", "./bad-prog.raku";
}

The "my" line by itself - it doesn't die, and as you've noted, if you don't do the assignment, it does die.

You can force the sink (and the exception) by explicitly using sink (but in these examples, I'd just use the simpler first version)

use Test;
dies-ok {
    sink my $p = run "-I", "./bad-prog.raku";
}

When you're assigning it, we're assuming you're going to do something with it, and so you get a Proc that you can introspect (that doesn't throw)

[coke]

Note that seems to contradict the existing docs.

Can you point at the docs at issue here?

[tbrowder]

    run "-I.", "./bad-prog.raku";

To be clear, this is trying to run a program named "-I" - it looks like you're expecting this to run "raku" ?

Yes, that was an error here, which I just corrected (along with the other test file). My test files on my PC are correct and did show the error reported.

[coke]

again, which docs in particular? The sink behavior is already documented in run, not sure it makes sense to add this to dies-ok.

[tbrowder]

I'll put together a detailed response tomorrow.

[tbrowder]

First, I corrected my original input by adding the shebang line in the bad prog so I could remove the "-I." line in the several "run..." lines.

The main problem I have with the current docs is that when I see the raku response it says "# Failed test at...line 2" with no hint of why, and the docs for "dies {}" says nothing about sinking in conjunction with 'run'.

When I look at the docs for "run" it says:

A sunk Proc object for a processs that exited...

    run "false";     # SUNK! Will throw
    run("false").so; # OK. Evaluates Proc...no sinking

That description seems in conflict with my observed behavior of "dies-ok" which says simply:

Marks a test as passed if the given code throws an exception.

Obviously that is misleading for my examples. I think there needs to be a bit more information there. Maybe add a link to appropriate parts of 'Proc' and 'sink'.

[ugexe]

Marks a test as passed if the given code throws an exception.

But your original code is not throwing an exception - a "bad proc" (by what I understand you to mean) returns something closer to a Failure. Sinking that failure is one way to make it turn into a thrown exception.

[tbrowder]

I think that's right, but my pea brain doesn't see that clearly enough to figure out what was happening originally, and the docs for "dies-ok {}" don't help.

[tbrowder]

On Tue, Apr 22, 2025 at 09:56 Nick Logan @.***> wrote:

Marks a test as passed if the given code throws an exception.

But your original code is not throwing an exception - a "bad proc" (by what I understand you to mean) returns a failure. Sinking that failure is one way to make it turn into a thrown exception.

— Reply to this email directly, view it on GitHub https://github.com/Raku/doc/issues/4569#issuecomment-2821608610, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAH5VVEOQHB3KQJKNP75SHT22ZKBJAVCNFSM6AAAAAB3ORXJ2WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQMRRGYYDQNRRGA . You are receiving this because you were assigned.Message ID: @.***> ugexe left a comment (Raku/doc#4569) https://github.com/Raku/doc/issues/4569#issuecomment-2821608610

Marks a test as passed if the given code throws an exception.

But your original code is not throwing an exception - a "bad proc" (by what I understand you to mean) returns a failure. Sinking that failure is one way to make it turn into a thrown exception.

Then should "eval-dies-ok {}" work for my "bad proc"example?

[tbrowder]

I see I don't understand enough to continue. But I still think the docs in the vicinity of "dies-ok" and "eval-dies-ok" need improvement for the non-expert Raku coder. "dies-ok" is such a handy tool for simple test-driven development that it needs a more detailed description.

One more thing, the silly examples for the tests deflect and confuse the basic lessons to be learned on how to use them.

Maybe I'll experiment with the "no fail" pragma (which I just discovered) inside a dies-ok block.

[coke]

eval-dies-ok isn't going to change the core issue here - when you invoke run in sink context, it throws an exception. dies-ok and eval-dies-ok are working as you intend with code that throws an exception. This context is tied to the code inside the block, not the use of dies-ok itself.

From the dies-ok docs:

Marks a test as passed if the given $code throws an exception.

From the run docs:

A sunk Proc object for a process that exited unsuccessfully will throw. If you wish to ignore such failures, simply use run in non-sink context:

As noted previously, one of your examples is in sink context, and one is not, and each one is working correctly with dies-ok.

For slightly more clarity in run's docs, I do suggest changing the quoted text here to specifically say "throw an Exception", and reword to avoid mention of "failure" because that could be confused with Failure.

[coke]

Maybe I'll experiment with the "no fail" pragma (which I just discovered) inside a dies-ok block.

What's this?

[lizmat]

$ raku -e 'no fail'
===SORRY!===
Don't know how to 'no fail'

Feels like it is a way to always die at compilation?

[ugexe]

They are talking about use fatal, and I'm not sure if that would work here since process failures are not a traditional Failure

[tbrowder]

What about, in the text following dies-ok {}, say something like:

The ‘run’ behavior in code
blocks depends on how it is
structured. See more details
in the descriptions in ‘run’
as well as ‘sink’, ‘Exception’
and ‘Failure’.

(with appropriate doc links for each term).

[coke]

That doesn't belong with dies-ok. It belongs with run, and it's already there. I don't want to add something similar to run because:

• then we're duplicating the information about run in two places.
• we should then duplicate it wherever something might catch an exception - e.g. in the docs for CATCH
• we should also duplicate docs wherever some other routine might throw or not throw.

[tbrowder]

I do not see a link to 'sink' or 'Exception' or 'Fatal' in the 'run' description text.

[coke]

Failure is not relevant to run. It's returning a Proc if it doesn't throw. There are other cases in Raku where you might get a Failure vs. an Exception.

Here's the text, which I quoted above:

A sunk [Proc](https://docs.raku.org/type/Proc) object for a process that [exited](https://docs.raku.org/routine/exitcode) unsuccessfully will throw. If you wish to ignore such failures, simply use [run](https://docs.raku.org/routine/run) in non-sink context:

Also as noted above, this could be made more explicit by rewording "throw" to "throw an Exception", linked to the exception type. But that's the only thing you can throw: Exceptions. And the docs in dies-ok say

throws an exception

So they're both showing the same text: "throw".

I've updated both snippets in the repo to mention Exception and link it to the overall type.

[coke]

I also added one more example to the run "sink or not sink" examples.

[wayland]

How about this as an idea:

• We note on the Catching Exceptions page that some functions may throw or not throw depending on context (and maybe link to examples; or not)
• We link to that part of Catching Exceptions from dies-ok with a note that "This follows the same exception-catching rules as CATCH" or something

Not sure if that's the best solution, but the responses to this might highlight some important point.

[coke]

I think I’ve addressed the issues as best as possible without a lot of duplication. Happy to consider a PR if someone has some better suggestions.