2.7.0: incorrect color with terminal background image

[i0ntempest]

In version 2.7.0 with Apple Terminal, background color seems to be broken, which can be seen if background image or transparency is enabled:

Debug log:

Please post the following report at <https://github.com/walles/moor/issues>,
or e-mail it to [email protected].

Version      : 2.7.0
LANG         : en_US.UTF-8
TERM         : xterm-256color
MOOR         : 
EDITOR       : /opt/local/bin/nano
TERM_PROGRAM : Apple_Terminal

LESS_TERMCAP_md : ESC[1;31m
LESS_TERMCAP_us : ESC[1;4;32m
LESS_TERMCAP_so : ESC[01;33m

GOOS    : darwin
GOARCH  : arm64
Compiler: gc
NumCPU  : 16

Stdin  is a terminal: true
Stdout is a terminal: true

time="Oct 25 20:07:08.051573" level=debug msg="File is assumed to be uncompressed: /Users/Admin/Documents/Scripts/auto-updatedb.sh"
time="Oct 25 20:07:08.054620" level=debug msg="File is assumed to be uncompressed: /Users/Admin/Documents/Scripts/auto-updatedb.sh"
time="Oct 25 20:07:08.054629" level=info msg="ttyin terminal state: &{state:{termios:{Iflag:26624 Oflag:2 Cflag:19200 Lflag:536870979 Cc:[4 255 255 127 23 21 18 255 3 28 26 25 17 19 22 15 1 0 20 255] Ispeed:9600 Ospeed:9600}}}"
time="Oct 25 20:07:08.054634" level=info msg="ttyout terminal state: &{state:{termios:{Iflag:26624 Oflag:2 Cflag:19200 Lflag:536870979 Cc:[4 255 255 127 23 21 18 255 3 28 26 25 17 19 22 15 1 0 20 255] Ispeed:9600 Ospeed:9600}}}"
time="Oct 25 20:07:08.054637" level=debug msg="Counted 13 lines in 7.375µs at 567ns/line"
time="Oct 25 20:07:08.054649" level=info msg="No known terminal with arrow keys emulation detected, assuming mouse tracking is needed"
time="Oct 25 20:07:08.054657" level=info msg="Stream read in 14.541µs, have 13 lines"
time="Oct 25 20:07:08.054674" level=info msg="Entering Twin main loop..."
time="Oct 25 20:07:08.055056" level=debug msg="Terminal background color detected as #808080 after 397.625µs"
time="Oct 25 20:07:08.060302" level=debug msg="Using style <tango>"
time="Oct 25 20:07:08.060306" level=info msg="Pager starting"
time="Oct 25 20:07:08.060453" level=info msg="Entering pager main loop..."
time="Oct 25 20:07:08.061599" level=debug msg="highlightFromMemory() took 6.937291ms"
time="Oct 25 20:07:08.061602" level=debug msg="Tailing file /Users/Admin/Documents/Scripts/auto-updatedb.sh"
time="Oct 25 20:07:23.861751" level=info msg="ttyin read error, twin giving up: EOF"

Apparantly everything between 2.1.1 and 2.7 is broken. 2.1.1 works fine:

Debug log:

Please post the following report at <https://github.com/walles/moor/issues>,
or e-mail it to [email protected].

Version      : 2.1.1
LANG         : en_US.UTF-8
TERM         : xterm-256color
MOOR         : 
EDITOR       : /opt/local/bin/nano
TERM_PROGRAM : Apple_Terminal

GOOS    : darwin
GOARCH  : arm64
Compiler: gc
NumCPU  : 16

Stdin  is a terminal: true
Stdout is a terminal: true

time="Oct 25 20:08:39.968928" level=debug msg="File is assumed to be uncompressed: /Users/Admin/Documents/Scripts/auto-updatedb.sh"
time="Oct 25 20:08:39.971657" level=debug msg="File is assumed to be uncompressed: /Users/Admin/Documents/Scripts/auto-updatedb.sh"
time="Oct 25 20:08:39.971668" level=debug msg="Counted 13 lines in 3.292µs at 253ns/line"
time="Oct 25 20:08:39.971674" level=info msg="Stream read in 3.375µs, have 13 lines"
time="Oct 25 20:08:39.971776" level=info msg="ttyin terminal state: &{state:{termios:{Iflag:26624 Oflag:2 Cflag:19200 Lflag:536870979 Cc:[4 255 255 127 23 21 18 255 3 28 26 25 17 19 22 15 1 0 20 255] Ispeed:9600 Ospeed:9600}}}"
time="Oct 25 20:08:39.971782" level=info msg="ttyout terminal state: &{state:{termios:{Iflag:26624 Oflag:2 Cflag:19200 Lflag:536870979 Cc:[4 255 255 127 23 21 18 255 3 28 26 25 17 19 22 15 1 0 20 255] Ispeed:9600 Ospeed:9600}}}"
time="Oct 25 20:08:39.971799" level=info msg="No known terminal with arrow keys emulation detected, assuming mouse tracking is needed"
time="Oct 25 20:08:39.971854" level=info msg="Entering Twin main loop..."
time="Oct 25 20:08:39.972390" level=debug msg="Terminal background color detected as #808080 after 568.041µs"
time="Oct 25 20:08:39.972394" level=debug msg="Using style <tango>"
time="Oct 25 20:08:39.972396" level=info msg="Pager starting"
time="Oct 25 20:08:39.972415" level=info msg="Entering pager main loop..."
time="Oct 25 20:08:39.973374" level=debug msg="highlightFromMemory() took 1.69775ms"
time="Oct 25 20:08:39.973377" level=debug msg="Tailing file /Users/Admin/Documents/Scripts/auto-updatedb.sh"

[walles]

Workaround

Set your MOOR environment variable to --terminal-fg.

Fix

moor needs to be able to tell that the terminal has a background image, and I don't know if there's a way to query for that.

If you have any ideas I'm all ears.

One thing that's happening though is that Apple's Terminal claims to have a background color:

time="Oct 25 20:08:39.972390" level=debug msg="Terminal background color detected as #808080 after 568.041µs"

Since there is no background color (but an image), this response from the terminal is not really true.

AFAIU, what the terminal should be doing is to not respond at all to moor's background query (since there is no background color).

Which would enable moor to adapt.

So maybe this needs to be brought up with Apple?

Notes

Just tested with iTerm2 and it works the same way.

[walles]

Posted a question here: https://stackoverflow.com/questions/79799431/how-to-determine-that-the-terminal-has-a-background-image-not-just-color

[i0ntempest]

Thanks, with the workaround moor is now usable. But there's still a few spots with incorrect background: Section headers when using moor as MANPAGER:

And line numbers:

Indeed seems like an Apple problem. Apple Terminal has some limitations compared to other third party terminals, I'm kinda considering switching just haven't made my mind yet.

[walles]

Did you set your own section headers using LESS_TERMCAP_* or are these from moor?

You can do moor --help to find out.

If you set them yourself you should consider not setting a background for them.

[i0ntempest]

Those are from my .profile:

export LESS_TERMCAP_mb=$'\e[1;31m'
export LESS_TERMCAP_md=$'\e[1;31m'
export LESS_TERMCAP_me=$'\e[0m'
export LESS_TERMCAP_se=$'\e[0m'
export LESS_TERMCAP_so=$'\e[01;33m'
export LESS_TERMCAP_ue=$'\e[0m'
export LESS_TERMCAP_us=$'\e[1;4;32m'

I don't think any of these sets background. moor --help gives:

And if I remove them man page would look like this:

[walles]

I just released v2.7.1 with updates to --terminal-fg, so your man pages should now look good with that option: https://github.com/walles/moor/releases/

On an unrelated note, moor automatically hides line numbers for man pages (and git output), so no need for --no-linenumbers in your MANPAGER environment variable. Also, man pages come pre wrapped, so I don't think --wrap has much effect in there.

You could consider dropping MANPAGER and setting these instead:

PAGER=moor
MOOR=--wrap

Also maybe consider --reformat for automatic JSON reformatting if that's something you're interested in, I find it super nice!

[i0ntempest]

Yup, man pages now look good. As for the line number though, I'm not sure why but they aren't auromatically removed when using in man pages so I have to use that arg.

[walles]

As for the line number though, I'm not sure why but they aren't auromatically removed when using in man pages

Strange, I wonder why this code wouldn't work in your case?

https://github.com/walles/moor/blob/83d6d54c5c92f3029a1d5bf0720208c81a4f0e5b/cmd/moor/moor.go#L330-L362

[i0ntempest]

Not sure, probably because I'm using mandoc from MacPorts

[walles]

Not sure, probably because I'm using mandoc from MacPorts

I want to detect that thing if it's easy enough.

Can you try doing this and sending me any lines with MAN in them?

Note that piping through grep won't work, you'll have to copy the result from the terminal and filter it afterwards.

PAGER=env MANPAGER=env man man

[i0ntempest]

Not sure, probably because I'm using mandoc from MacPorts

I want to detect that thing if it's easy enough.

Can you try doing this and sending me any lines with MAN in them?

Note that piping through grep won't work, you'll have to copy the result from the terminal and filter it afterwards.

PAGER=env MANPAGER=env man man

Doesn't seem to work:

env: /tmp/man.2xTiqBv5Ck: Permission denied

This is probably due to how mandoc works (different from man-db). PAGER=env MANPAGER=env /usr/bin/man man does work but the output isn't useful.

Also found another minor color bug while doing a search, all lines containing a match will have incorrect background:

[walles]

I looked into mandoc and it doesn't seem to do anything that I can detect. Not related to this issue really, but since we side-tracked into there I wanted to note my attempt somewhere.

[walles]

Actually, in 2.9.1 I started detecting man pages based on their formatting, so on that side track you should now be covered.