teemtee
`tmt --no-color` prints color on `discover` failure
$ tmt --no-color run discover -h fmf -u https://127.0.0.1/nonexistent 2>&1 | cat -A
/var/tmp/tmt/run-089$
$
/plans/example$
discover$
how: fmf$
url: https://127.0.0.1/nonexistent$
$
^[[31mplan failed^[[0m$
$
The exception was caused by 1 earlier exceptions$
$
Cause number 1:$
$
^[[31mCommand 'git clone https://127.0.0.1/nonexistent /var/tmp/tmt/run-089/plans/example/discover/default-0/tests' returned 128.^[[0m$
$
stderr (2/2 lines)$
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~$
Cloning into '/var/tmp/tmt/run-089/plans/example/discover/default-0/tests'...$
fatal: unable to access 'https://127.0.0.1/nonexistent/': Failed to connect to 127.0.0.1 port 443 after 0 ms: Connection refused$
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~$
$
Note the special color characters on plan failed and on the Command ... returned ... lines.
FTR, it's caused by the show_exception() and render_exception() ignoring the colorization settings. They don't depend on any logger, and they may get called very early, and they use click.style(). It is desirable to use colors to highlight exceptions and tracebacks, but we will need to make them optional, an addition that can be enabled once main() decides whether they are allowed. Until then, stick to no colors.
Hi @happz and @lukaszachy, looks we cannot add a simple test to this issue because the color char is gone if we save the stdout to a file when running tmt run discover -h fmf -u https://127.0.0.1/nonexistent | cat -A. e.g.
(dev) huanli@kvm-01-guest09:discover$ cat exception.sh
#!/bin/bash
. /usr/share/beakerlib/beakerlib.sh || exit 1
rlJournalStart
rlPhaseStartSetup
rlRun "pushd order"
rlRun "tmp=\$(mktemp -d)" 0 "Creating tmp directory"
rlPhaseEnd
rlPhaseStartTest "Test exception without option '--no-color'"
tmt run discover -h fmf -u https://127.0.0.1/nonexistent | cat -A
rlPhaseEnd
rlPhaseStartCleanup
rlRun 'rm -rf $tmp' 0 "Remove tmp directory"
rlPhaseEnd
rlJournalEnd
(dev) huanli@kvm-01-guest09:discover$ ./exception.sh
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Setup
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: [ 17:08:27 ] :: [ BEGIN ] :: Running 'pushd order'
~/dev/2759/dev/tests/discover/order ~/dev/2759/dev/tests/discover
:: [ 17:08:27 ] :: [ PASS ] :: Command 'pushd order' (Expected 0, got 0)
:: [ 17:08:27 ] :: [ BEGIN ] :: Creating tmp directory :: actually running 'tmp=$(mktemp -d)'
:: [ 17:08:27 ] :: [ PASS ] :: Creating tmp directory (Expected 0, got 0)
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Duration: 0s
:: Assertions: 2 good, 0 bad
:: RESULT: PASS (Setup)
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Test exception without option '--no-color'
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
/var/tmp/tmt/run-027
/plans/multiple-by-enumerate
discover
how: fmf
name: enumerate-and-order
url: https://127.0.0.1/nonexistent
fail: Command 'git clone https://127.0.0.1/nonexistent /var/tmp/tmt/run-027/plans/multiple-by-enumerate/discover/enumerate-and-order/tests' returned 128.
fail: Command 'git clone https://127.0.0.1/nonexistent /var/tmp/tmt/run-027/plans/multiple-by-enumerate/discover/enumerate-and-order/tests' returned 128.
fail: Command 'git clone https://127.0.0.1/nonexistent /var/tmp/tmt/run-027/plans/multiple-by-enumerate/discover/enumerate-and-order/tests' returned 128.
$
plan failed$
$
The exception was caused by 1 earlier exceptions$
$
Cause number 1:$
$
Retries of 'git clone https://127.0.0.1/nonexistent /var/tmp/tmt/run-027/plans/multiple-by-enumerate/discover/enumerate-and-order/tests' unsuccessful.$
$
The exception was caused by 3 earlier exceptions$
$
Cause number 1:$
$
Command 'git clone https://127.0.0.1/nonexistent /var/tmp/tmt/run-027/plans/multiple-by-enumerate/discover/enumerate-and-order/tests' returned 128.$
$
stderr (2/2 lines)$
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~$
Cloning into '/var/tmp/tmt/run-027/plans/multiple-by-enumerate/discover/enumerate-and-order/tests'...$
fatal: unable to access 'https://127.0.0.1/nonexistent/': Failed to connect to 127.0.0.1 port 443 after 0 ms: Couldn't connect to server$
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~$
$
$
Cause number 2:$
$
Command 'git clone https://127.0.0.1/nonexistent /var/tmp/tmt/run-027/plans/multiple-by-enumerate/discover/enumerate-and-order/tests' returned 128.$
$
stderr (2/2 lines)$
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~$
Cloning into '/var/tmp/tmt/run-027/plans/multiple-by-enumerate/discover/enumerate-and-order/tests'...$
fatal: unable to access 'https://127.0.0.1/nonexistent/': Failed to connect to 127.0.0.1 port 443 after 0 ms: Couldn't connect to server$
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~$
$
$
Cause number 3:$
$
Command 'git clone https://127.0.0.1/nonexistent /var/tmp/tmt/run-027/plans/multiple-by-enumerate/discover/enumerate-and-order/tests' returned 128.$
$
stderr (2/2 lines)$
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~$
Cloning into '/var/tmp/tmt/run-027/plans/multiple-by-enumerate/discover/enumerate-and-order/tests'...$
fatal: unable to access 'https://127.0.0.1/nonexistent/': Failed to connect to 127.0.0.1 port 443 after 0 ms: Couldn't connect to server$
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~$
...<snip>...
If we cannot verify the color characters when running tmt run discover -h fmf -u https://127.0.0.1/nonexistent, it makes no sense to check the output of tmt --no-color run discover -h fmf -u https://127.0.0.1/nonexistent contains colored characters.
Any idea/suggestion?
@idorax Why tmt run discover -h fmf -u https://127.0.0.1/nonexistent | cat -A ? It was used to illustrate the problem.
Could you please try again with rlRun -s "tmt run ...." and grep the file at $rlRun_LOG for the color control chars?
BTW I see red chars in the github render of the file you posted (first ~ of some lines).
Hi @lukaszachy,
Could you please try again with rlRun -s "tmt run ...." and grep the file at $rlRun_LOG for the color control chars?
Will have a try and get back to you.
BTW I see red chars in the github render of the file you posted (first ~ of some lines).
It is because I used
```bash
to post the related text. e.g.
- without ```bash
:: [ 17:08:27 ] :: [ BEGIN ] :: Running 'pushd order'
~/dev/2759/dev/tests/discover/order ~/dev/2759/dev/tests/discover
- with ```bash
:: [ 17:08:27 ] :: [ BEGIN ] :: Running 'pushd order'
~/dev/2759/dev/tests/discover/order ~/dev/2759/dev/tests/discover
Indeed, rlRun -S "..." and then checking $rlRunLOG seems to work as @lukaszachy described.
Indeed, rlRun -S "..." and then checking $rlRunLOG seems to work as @lukaszachy described.
Hi @happz, looks rlRun doesn't have -S. e.g.
(dev) huanli@kvm-01-guest09:discover$ cat -n exception.sh | head -20
1 #!/bin/bash
2
3 . /usr/share/beakerlib/beakerlib.sh || exit 1
4
5 rlJournalStart
6 rlPhaseStartSetup
7 rlRun "pushd order"
8 rlRun "tmp=\$(mktemp -d)" 0 "Creating tmp directory"
9 rlPhaseEnd
10
11 rlPhaseStartTest "Test exception without option '--no-color'"
12 rlRun -S "tmt run discover -h fmf -u https://127.0.0.1/nonexistent" "1-255"
13 cat -A $rlRun_LOG
14 echo $rlRun_LOG
15 cp $rlRun_LOG /tmp/a
16 #rlAssertGrep "^[[31m" $rlRun_LOG
17 rlPhaseEnd
18
19 # rlPhaseStartTest "Test exception with option '--no-color'"
20 # rlRun -s "tmt --no-color run discover -h fmf -u https://127.0.0.1/nonexistent" "1-255"
(dev) huanli@kvm-01-guest09:discover$ ./exception.sh
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Setup
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: [ 11:40:23 ] :: [ BEGIN ] :: Running 'pushd order'
~/dev/2759/dev/tests/discover/order ~/dev/2759/dev/tests/discover
:: [ 11:40:23 ] :: [ PASS ] :: Command 'pushd order' (Expected 0, got 0)
:: [ 11:40:23 ] :: [ BEGIN ] :: Creating tmp directory :: actually running 'tmp=$(mktemp -d)'
:: [ 11:40:23 ] :: [ PASS ] :: Creating tmp directory (Expected 0, got 0)
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Duration: 1s
:: Assertions: 2 good, 0 bad
:: RESULT: PASS (Setup)
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Test exception without option '--no-color'
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: [ 11:40:23 ] :: [ ERROR ] :: rlRun: getopt: invalid option -- 'S'
Do you mean rlRun -s "..."?
Hi @lukaszachy and @happz, the command rlRun -s "tmt run discover -h fmf -u https://127.0.0.1/nonexistent" "1-255" failed to print out the color chars ^[[31m. Would you please help to have a try with https://github.com/teemtee/tmt/pull/2816/commits/d51965ac68d834c52d9b4ec564a699af9e0e4ab2 in your development environment? Thanks!
@idorax indeed, it did not work when I tried. tmt detected it's running behind a non-tty, and turned colors off. Meh.
Anyway, I played with it a bit, and I ended up with the following steps:
- use ptty wrapper from
tests/execute/tty, - fixed the ptty wrapper to propagate exit code,
- perl-mode grep & a pattern for control character,
and it worked eventually:
diff --git a/tests/discover/exception.sh b/tests/discover/exception.sh
index 3758faee..58cb0e57 100755
--- a/tests/discover/exception.sh
+++ b/tests/discover/exception.sh
@@ -2,22 +2,23 @@
. /usr/share/beakerlib/beakerlib.sh || exit 1
+PTTY_WRAPPER="../execute/tty/ptty-wrapper"
+
rlJournalStart
rlPhaseStartSetup
- rlRun "pushd order"
rlRun "tmp=\$(mktemp -d)" 0 "Creating tmp directory"
rlPhaseEnd
rlPhaseStartTest "Test exception without option '--no-color'"
- rlRun -s "tmt run discover -h fmf -u https://127.0.0.1/nonexistent" "1-255"
+ rlRun -s "$PTTY_WRAPPER tmt run --id $tmp --scratch discover -h fmf -u https://127.0.0.1/nonexistent" "1-255"
cat -A $rlRun_LOG
- #rlAssertGrep "^[[31m" $rlRun_LOG
+ rlRun "grep -aP '\e\[31m' $rlRun_LOG"
rlPhaseEnd
rlPhaseStartTest "Test exception with option '--no-color'"
- rlRun -s "tmt --no-color run discover -h fmf -u https://127.0.0.1/nonexistent" "1-255"
+ rlRun -s "$PTTY_WRAPPER tmt --no-color run --id $tmp --scratch discover -h fmf -u https://127.0.0.1/nonexistent" "1-255"
cat -A $rlRun_LOG
- #rlAssertNotGrep "^[[31m" $rlRun_LOG
+ rlRun "grep -aP '\e\[31m' $rlRun_LOG" 1
rlPhaseEnd
rlPhaseStartCleanup
diff --git a/tests/execute/tty/ptty-wrapper b/tests/execute/tty/ptty-wrapper
index 969e87ed..2b3da822 100755
--- a/tests/execute/tty/ptty-wrapper
+++ b/tests/execute/tty/ptty-wrapper
@@ -8,4 +8,5 @@ import pty
import sys
os.environ["TERM"] = "vt100"
-pty.spawn(("/bin/bash", "-c", " ".join(sys.argv[1:])))
+exit_status = pty.spawn(("/bin/bash", "-c", " ".join(sys.argv[1:])))
+sys.exit(os.waitstatus_to_exitcode(exit_status))