`fish` shell's `--help` command is not highlighted

[KaBankz]

description

The fish shell's help command output is not highlighted at all.

When piping into bat:

Regular:

$ bat --version

bat 0.24.0

$ fish --help

fish - the friendly interactive shell

ffiisshh [_O_P_T_I_O_N_S] [_F_I_L_E [_A_R_G ...]]
ffiisshh [_O_P_T_I_O_N_S] [--cc _C_O_M_M_A_N_D [_A_R_G ...]]

DDEESSCCRRIIPPTTIIOONN
ffiisshh is a command-line shell written mainly with interactive use in
mind.  This page briefly describes the options for invoking ffiisshh.
The _f_u_l_l _m_a_n_u_a_l is available in HTML by using the hheellpp command from
inside fish, and in the _f_i_s_h_-_d_o_c_(_1_) man page.  The _t_u_t_o_r_i_a_l is
available as HTML via hheellpp ttuuttoorriiaall or in _m_a_n _f_i_s_h_-_t_u_t_o_r_i_a_l.

The following options are available:

--cc or ----ccoommmmaanndd==CCOOMMMMAANNDD
       Evaluate the specified commands instead of reading from the
       commandline, passing additional positional arguments through
       $$aarrggvv.

--CC or ----iinniitt--ccoommmmaanndd==CCOOMMMMAANNDDSS
       Evaluate specified commands after reading the configuration
       but before executing command specified by --cc or reading
       interactive input.

--dd or ----ddeebbuugg==DDEEBBUUGG__CCAATTEEGGOORRIIEESS
       Enables debug output and specify a pattern for matching debug
       categories.  See _D_e_b_u_g_g_i_n_g below for details.

--oo or ----ddeebbuugg--oouuttppuutt==DDEEBBUUGG__FFIILLEE
       Specifies a file path to receive the debug output, including
       categories and  _f_i_s_h___t_r_a_c_e.  The default is stderr.

--ii or ----iinntteerraaccttiivvee
       The shell is interactive.

--ll or ----llooggiinn
       Act as if invoked as a login shell.

--NN or ----nnoo--ccoonnffiigg
       Do not read configuration files.

--nn or ----nnoo--eexxeeccuuttee
       Do not execute any commands, only perform syntax checking.

--pp or ----pprrooffiillee==PPRROOFFIILLEE__FFIILLEE
       when ffiisshh exits, output timing information on all executed
       commands to the specified file.  This excludes time spent
       starting up and reading the configuration.

----pprrooffiillee--ssttaarrttuupp==PPRROOFFIILLEE__FFIILLEE
       Will write timing for ffiisshh startup to specified file.

--PP or ----pprriivvaattee
       Enables _p_r_i_v_a_t_e _m_o_d_e: ffiisshh will not access old or store new
       history.

----pprriinntt--rruussaaggee--sseellff
       When ffiisshh exits, output stats from getrusage.

----pprriinntt--ddeebbuugg--ccaatteeggoorriieess
       Print all debug categories, and then exit.

--vv or ----vveerrssiioonn
       Print version and exit.

--ff or ----ffeeaattuurreess==FFEEAATTUURREESS
       Enables one or more comma-separated _f_e_a_t_u_r_e _f_l_a_g_s.

The ffiisshh exit status is generally the _e_x_i_t _s_t_a_t_u_s _o_f _t_h_e _l_a_s_t
_f_o_r_e_g_r_o_u_n_d _c_o_m_m_a_n_d.

DDEEBBUUGGGGIINNGG
While fish provides extensive support for _d_e_b_u_g_g_i_n_g _f_i_s_h _s_c_r_i_p_t_s, it
is also possible to debug and instrument its internals.  Debugging
can be enabled by passing the ----ddeebbuugg option.  For example, the
following command turns on debugging for background IO thread events,
in addition to the default categories, i.e. _d_e_b_u_g, _e_r_r_o_r, _w_a_r_n_i_n_g,
and _w_a_r_n_i_n_g_-_p_a_t_h:

   > fish --debug=iothread

Available categories are listed by ffiisshh ----pprriinntt--ddeebbuugg--ccaatteeggoorriieess. The
----ddeebbuugg option accepts a comma-separated list of categories, and
supports glob syntax.  The following command turns on debugging for
_c_o_m_p_l_e_t_e, _h_i_s_t_o_r_y, _h_i_s_t_o_r_y_-_f_i_l_e, and _p_r_o_f_i_l_e_-_h_i_s_t_o_r_y, as well as the
default categories:

   > fish --debug='complete,*history*'

Debug messages output to stderr by default. Note that if _f_i_s_h___t_r_a_c_e
is set, execution tracing also outputs to stderr by default. You can
output to a file using the ----ddeebbuugg--oouuttppuutt option:

   > fish --debug='complete,*history*' --debug-output=/tmp/fish.log --init-command='set fish_trace on'

These options can also be changed via the _F_I_S_H___D_E_B_U_G and
_F_I_S_H___D_E_B_U_G___O_U_T_P_U_T variables.  The categories enabled via ----ddeebbuugg are
_a_d_d_e_d to the ones enabled by $FISH_DEBUG, so they can be disabled by
prefixing them with -- (rreeaaddeerr--**,,--aasstt** enables reader debugging and
disables ast debugging).

The file given in ----ddeebbuugg--oouuttppuutt takes precedence over the file in
_F_I_S_H___D_E_B_U_G___O_U_T_P_U_T.

$ fish --version

3.7.1

[victor-gp]

Thanks for reporting this @KaBankz! Unfortunately, this kind of help message is intractable for cmd-help.

It seems like fish's help message uses overstrike formatting [1]. Notice in the copypasted help message how some lines have lots of

char X + mystery char ? + char X again

I'm guessing the unknown char is a backspace, which checks out with how this combination is represented as a bold char X (see the screenshots).

It'd be super complicated to handle this with a sublime syntax, so I decided early on [2] that cmd-help wouldn't cover this use case...

I appreciate your reporting this in any case! Also, if cmd-help was useful to you please give it a star!

1. https://unix.stackexchange.com/a/118709/545324
2. https://github.com/victor-gp/cmd-help-sublime-syntax/commit/055fe34acc1eafaa739c649805b1e19afa8e653e

[KaBankz]

Unfortunate for fish, but thanks anyway for this cool tool, it's made my help usage much more pleasant.

[victor-gp]

Hi @KaBankz !

I've given this another look and maybe we can strip special characters from the input before it reaches the syntax.

Can you try running this?

fish --help | sed -r "s,\x1B\[[0-9;]*[mK],,g" | bat -plhelp

You could later put that sed command into an alias, perhaps, or the entire | sed ... | bat ... line.

If it works, I'll probably add it to the suggested configuration in the docs.

That particular sed command comes from this StackOverflow answer.

If you're on a BSD system (e.g.: MacOS), you may need to try this one.

[KaBankz]

Hey @victor-gp,

The sed commands you provided didn't work, I'm on MacOS btw, and none of the sed commands from the linked StackOverflow worked for me.

I did some digging myself and no luck, but then I decided to ask ChatGPT, and low and behold, it gave me:

sed 's/.\x08//g'

.\x08: represents the backspace character, this is typical of overstriking where one character overlays another. ― ChatGPT

This seems to work perfectly to convert the overstriking formatted output to plaintext

Before:

fish - the friendly interactive shell

ffiisshh [_O_P_T_I_O_N_S] [_F_I_L_E [_A_R_G ...]]
ffiisshh [_O_P_T_I_O_N_S] [--cc _C_O_M_M_A_N_D [_A_R_G ...]]

After:

fish - the friendly interactive shell

fish [OPTIONS] [FILE [ARG ...]]
fish [OPTIONS] [-c COMMAND [ARG ...]]

Now using it with bat -lhelp works as expected

fish --help | sed 's/.\x08//g' | bat -lhelp

Although some nested stuff is not highlighted and some flags are highlighted while others are not, not sure if this is expected behavior or a bug: (sorry for the obnoxious arrows)

Other than that, this solution seems to work perfect for fish!

TL;DR 🎉 This work's for fish 🎉

fish --help | sed 's/.\x08//g' | bat -lhelp

[victor-gp]

Hi @KaBankz, sorry for the radio silence! Life got in the way, and then I kept putting this off because I had to write docs. Documentation is hard! :weary:

That said, I really appreciate your coming up with a solution. Not only does it work for your OSX + Fish setup, but also for my Ubuntu + Zsh one, and I guess every other shell. It's a general solution for help messages that use overstriking, like less.

So cheers to ChatGPT for the solution and to you for asking the right questions. :tada:

I also want you to know that I've compiled this and other useful tweaks into this Configuration Tips doc. It covers a few shell environments including Fish, so you may find a handy trick or two. ^^

I'll follow up in another comment regarding the highlighting inconsistencies you pointed out. I have to get my thoughts in order. :sweat_smile: But I'll say thank you in advance!

[victor-gp]

Firstly, thank you for the screenshot and the arrows. They were helpful!

I took the liberty of numbering the arrows to better group them into different issues:

(1) Arrows: --options in descriptions

These arrows highlight --options embedded in normal text. Right now, we're not targeting those for matters of complexity. Some reasons:

• It's much easier to target options, arguments, and subcommands when they appear at the start of a line (where they're being defined)...
• ...than when they appear mid-sentence in descriptions, mixed with plain text and separated by single spaces.
• While options might be easy enough to detect, other syntax elements (e.g., arguments, paths) would be very hard to get right.

Even if we could reliably highlight some forms of those elements (say, option arguments =LIKE_THIS), there would still be the issue of consistency:

• If we can only target some shapes of option arguments, should we colorize them at all?
• If we can only differentiate options, and not arguments or subcommands, should we colorize only options?

For these challenges, and perhaps more importantly to cut down on complexity, this is currently out of scope.

That said, there's a long-term goal in the Roadmap for an alternative version of the syntax that scopes tokens within descriptions.

I can't promise when/if I'll get to it, as there are plenty of things in "simple" cmd-help that should improve before that.

(2) Arrow: incorrectly highlighted --debug

Yep, this one's a bug!

--debug is part of the description text, so it definitely shouldn't be highlighted. I've added a syntax (unit) test to flag this and will check if it's easily fixed without breaking anything else.

(3) Arrows: example usage lines

These arrows point to example usages of the command. These are currently out of scope, but once the syntax can handle usage lines (#24), example lines should follow naturally.

I've also added a test case to track this.

Closing

In the meantime, I'll be closing this issue because the original problem is pretty much solved. You're welcome to open separate issues for any of the other points if you think they deserve more focused discussion.

Lastly, thank you very much! For the detailed report, the solution re: overstriking, and the care you put into this. 💜