bash-completion icon indicating copy to clipboard operation
bash-completion copied to clipboard

Published 20 hours ago • verified publisher icon scop

bash command completion has a problem with sudo command

Open mug896 opened this issue 2 years ago • 17 comments

Describe the bug

When the command completion function starts running there are three variables automatically setting $1, $2, $3. and $2 is not always the same as ${COMP_WORDS[COMP_CWORD]}.

  1. if $2 value is a member of $COMP_WORDBREAKS variable then it's value is empty by default

    ex) command --foo=[tab] $2 : (empty) ${COMP_WORDS[COMP_CWORD]} : =

  2. if $2 value start with " or ' (single or double quote) then it's quote removed ( "foo --> foo )

    ex) command --foo "bar[tab] $2 : bar ${COMP_WORDS[COMP_CWORD]} : "bar

this is an intended work for making completion function simpler but if i use sudo in front of a command then $2 value is always the same as ${COMP_WORDS[COMP_CWORD]} This is a different behavior to the bash default and problematic. and ends up can not use $2 variable and force to use ${COMP_WORDS[COMP_CWORD]} to support sudo

Versions

  • [ ] Operating system name/distribution and version: Operating System: Ubuntu 22.04.1 LTS
    Kernel: Linux 5.15.0-43-generic Architecture: x86-64
  • [ ] bash version: 5.1.16(1)-release
  • [ ] bash-completion version: 2.11

mug896 avatar Aug 23 '22 02:08 mug896

The call of the completion functions for the commands after another command (such as sudo) is emulated by _command_offset. In particular, the arguments passed to the completion function is determined here.

  • if $2 value is a member of $COMP_WORDBREAKS variable then it's value is empty by default ex) command --foo=[tab] $2 : (empty) ${COMP_WORDS[COMP_CWORD]} : =
  • if $2 value start with " or ' (single or double quote) then it's quote removed ( "foo --> foo ) ex) command --foo "bar[tab] $2 : bar ${COMP_WORDS[COMP_CWORD]} : "bar

Is this the exact rule? If the exact rule is known, I think we can implement it in _command_offset, but Bash Reference Manual doesn't seem to clarify the details.

Bash Reference Manual - §8.6 Programmable Completion

[...] When the function or command is invoked, the first argument ($1) is the name of the command whose arguments are being completed, the second argument ($2) is the word being completed, and the third argument ($3) is the word preceding the word being completed on the current command line. [...]

I have tried it a little with the plain Bash (5.1.16/Fedora 36), but the rule seems to be a bit more complicated. For special characters in COMP_WORDBREAKS (particularly = and :),

$ _test1() { printf '\e7\n\e[J'; declare -p COMP_WORDS COMP_CWORD; (($#)) && printf '<%s>' "$@"; printf '\e8'; } && complete -F _test1 test1
$ cat -A <<< "$COMP_WORDBREAKS"
 ^I$
"'@><=;|&(:$
$ test1 --foo==[TAB]
declare -a COMP_WORDS=([0]="test1" [1]="--foo" [2]="==")
declare -- COMP_CWORD="2"
<test1><><--foo>
$ test1 --foo=:[TAB]
declare -a COMP_WORDS=([0]="test1" [1]="--foo" [2]="=:")
declare -- COMP_CWORD="2"
<test1><><--foo>
$ test1 --foo'=[TAB]
declare -a COMP_WORDS=([0]="test1" [1]="--foo'=")
declare -- COMP_CWORD="1"
<test1><=><test1>
$ test1 --foo =[TAB]
declare -a COMP_WORDS=([0]="test1" [1]="--foo" [2]="=")
declare -- COMP_CWORD="2"
<test1><><--foo>
$ test1 --foo '=[TAB]
declare -a COMP_WORDS=([0]="test1" [1]="--foo" [2]="'=")
declare -- COMP_CWORD="2"
<test1><=><--foo>
$ test1 --foo '='[TAB]
declare -a COMP_WORDS=([0]="test1" [1]="--foo" [2]="'='")
declare -- COMP_CWORD="2"
<test1><'='><--foo>

For double quotation,

$ test1 a"b[TAB]
declare -a COMP_WORDS=([0]="test1" [1]="a\"b")
declare -- COMP_CWORD="1"
<test1><b><test1>
$ test1 a"b"[TAB]
declare -a COMP_WORDS=([0]="test1" [1]="a\"b\"")
declare -- COMP_CWORD="1"
<test1><a"b"><test1>
$ test1 a"b"c[TAB]
declare -a COMP_WORDS=([0]="test1" [1]="a\"b\"c")
declare -- COMP_CWORD="1"
<test1><a"b"c><test1>
$ test1 a"b"c"[TAB]
declare -a COMP_WORDS=([0]="test1" [1]="a\"b\"c\"")
declare -- COMP_CWORD="1"
<test1><><test1>

Also, $2 changes depending on the position of the cursor (where the cursor position is indicated by [TAB]):

$ test1 a[TAB]bcd
declare -a COMP_WORDS=([0]="test1" [1]="abcd")
declare -- COMP_CWORD="1"
<test1><a><test1>
$ test1 ab[TAB]cd
declare -a COMP_WORDS=([0]="test1" [1]="abcd")
declare -- COMP_CWORD="1"
<test1><ab><test1>
$ test1 abc[TAB]d
declare -a COMP_WORDS=([0]="test1" [1]="abcd")
declare -- COMP_CWORD="1"
<test1><abc><test1>
]$ test1 abcd[TAB]
declare -a COMP_WORDS=([0]="test1" [1]="abcd")
declare -- COMP_CWORD="1"
<test1><abcd><test1>

@mug896 Could you investigate and summarize the exact rule?

akinomyoga avatar Aug 23 '22 10:08 akinomyoga

I asked this problem to the current bash maintainer. and he said without bash-completion installed get the same null second argument with or without specifying 'sudo'

I think no needs to parse quotes directly In _sudo function just backup the $2, $3 variables and pass the values to the _hello function something like this

bash$ sudo hello aa bb cc

complete -F _sudo sudo
complete -F _hello hello

_sudo() {
	local two_bak=$2 three_bak=$3     # backup $2, $3
	. . .
	_sudo_sub
}
_sudo_sub() {
	. . .
	_hello "hello" "$two_bak" "$three_bak"
}

mug896 avatar Aug 23 '22 12:08 mug896

I think that would only work if we would just care about sudo, but we also need to care about completions for other commands that calls _command_offset. Also, to save the original arguments passed to the completion functions, we need to change the basic use pattern of bash-completion utility functions, which starts from the declaration of a pre-defined set of local variables and _init_completion. Furthermore, _command_offset and other utilities of bash-completion are called by external completions, i.e., custom completion functions written by users or third-party completion functions written by authors of applications, which makes it difficult to introduce breaking changes to the existing utility functions.

For these reasons, to follow your suggestion, we need to pay large efforts. We need to 1) design a new basic interface of bash-completion utilities for completion functions with new names or options, 2) rewrite all the existing completion functions to follow the new interface (there are currently 567 such functions in the repository), and 3) deprecate related existing functions or usages like _init_completion || return. We also need to 4) announce the interface changes to the authors of external completions and 5) submit pull requests if necessary. The question is whether it is worth making such large efforts to solve this issue. In particular, basically, we cannot change the basic interface used by external completions unless there is a sufficient reason. Even if we decided to break the existing interface, this would be a several-year project (including design discussion, implementations, tests, and reviews), and some continuous contributors need to voluntarily work on them, but at least I'm not willing to work on them.

akinomyoga avatar Aug 23 '22 14:08 akinomyoga

check this script out. i made this for fixing the bug of sudo completion function. and it works for me though i'm not using bash-completion package in my completion function. i think you should fix this bug anyway. because of this bug every completion function has to start with local cmd=$1 cur=${COMP_WORDS[COMP_CWORD]} prev=${COMP_WORDS[COMP_CWORD-1]}; [[ ${COMP_LINE:COMP_POINT-1:1} = " " ]] && cur="" and adds [[ $cur == "=" ]] && cur="" or [[ $cur == @(\"*|\'*) ]] && cur=${cur:1} lines instead of just writing local cmd=$1 cur=$2 prev=$3 to support bash-completion package

_sudo() 
{
    local arr func cmd=${COMP_WORDS[1]} cur=$2 i
    if (( COMP_CWORD == 1 )); then
        COMPREPLY=($( compgen -A command -- "$cur" ))
        return
    fi
    if ! complete -p "$cmd" &> /dev/null; then
        _completion_loader "$cmd" &> /dev/null
    fi
    if arr=( $(complete -p "$cmd" 2> /dev/null) ); then
        for (( i = 1; i < ${#arr[@]}; i++ )); do 
            [[ ${arr[i]} == -F ]] && { func=${arr[i + 1]}; break ;}
        done
        if [[ -n $func ]]; then 
            COMP_LINE=${COMP_LINE##$1+( )}
            let COMP_POINT-="COMP_POINT - ${#COMP_LINE}"
            unset -v 'COMP_WORDS[0]'
            COMP_WORDS=( "${COMP_WORDS[@]}" )
            let COMP_CWORD--
            "$func" "$cmd" "$2" "$3"
        fi
    fi
}

complete -o default -o bashdefault -F _sudo sudo

mug896 avatar Aug 24 '22 07:08 mug896

Thanks, but I know how to fix the issue if it is fine to fix only sudo as you are suggesting.

As I have explained in https://github.com/scop/bash-completion/issues/790#issuecomment-1224159227, the point is that we wouldn't like to fix the issue for each command like sudo, ionice, ccache, find, gdb, screen, strace, sudo, timeout, valgrind, watch, xvfb-run ... one by one separately. We should rather fix the utility function _command_offset because the implementation of _command_offset causes the present problem. The problem is that we cannot easily change the interface of _command_offset, so

  • One option is to implement the special rule to determine $2 from ${COMP_WORDS{COMP_CWORD]} in _command_offset. This is what I initially suggested. This solves the issue without requiring any changes to the completion functions in bash-completion and in external completions.
  • Another option is to save the original $2 passed by the readline to a fixed position and let _command_offset pick up from that position. However, this requires a change of the standard pattern local cur prev words cword split; _init_completion -s || return.
  • Another option is to add a new implementation of _command_offset2 or add a new option to _command_offset to specify a preferred $2 as an argument. However, this also involves the interface change so we need to rewrite all the calls of _command_offset. Of course, we can rewrite all such instances in the bash-completion project, but it is not the case for external completions.

akinomyoga avatar Aug 24 '22 08:08 akinomyoga

The problem is that the bash-completion package installed by default all linux distros. so even users not using the package have to use the sudo completion function in the package. The only command to concern is sudo command, not any other commands. and the sudo completion function in the package change $2 default value to ${COMP_WORDS[COMP_CWORD]}. I had some tests with the sudo function in the package and my sudo function above with ccache, find, gcc, strace, timeout, valgrind, watch ... commands. and they all work the same without problems because that commands use _init_completion || return. and not use $2 variable directly

mug896 avatar Aug 24 '22 13:08 mug896

The problem is that the bash-completion package installed by default all linux distros. so even users not using the package have to use the sudo completion function in the package.

I am not trying to say that the current bash-completion doesn't need to be fixed. You don't have to repeat in every your reply that the current behavior is a problem and it needs to be fixed. We are talking about the solution. As I have mentioned in my previous replies, the solution that you are suggesting has another maintenance problem though I know that it solves the original problem of sudo. You are repeating that your suggestion solves the original problem, but you don't seem to provide a solution to the maintenance problem. Unless you provide a satisfactory answer to the maintenance problem or consider other solutions, I think the discussion doesn't proceed.

The only command to concern is sudo command, not any other commands.

Why?

and the sudo completion function in the package change $2 default value to ${COMP_WORDS[COMP_CWORD]}.

These other commands are also affected by the current behavior of bash-completion because of the same problem in _command_offset. I won't create the test cases of every command for you, but that can be easily reproduced by, for example,

$ _test1() { printf '\e7\n\e[J'; printf '<%s>' "$@"; printf '\e8'; } && complete -F _test1 test1
$ test1 --foo=
<test1><><--foo>
$ sudo test1 --foo=
<test1><=><--foo>
$ ccache test1 --foo=
<test1><=><--foo>
$ find . -exec test1 --foo=
<test1><=><--foo>
$ gdb --args test1 --foo=
<test1><=><--foo>
$ strace test1 --foo=
<test1><=><--foo>

I had some tests with the sudo function in the package and my sudo function above with ccache, find, gcc, strace, timeout, valgrind, watch ... commands. and they all work the same without problems because that commands use _init_completion || return. and not use $2 variable directly

Could you describe in more detail how you implemented or patched the completion functions for those commands and how did you test them?

akinomyoga avatar Aug 25 '22 01:08 akinomyoga

I never thought of completions in the command line options that way. I'm sorry about my ignorance. my preferred way is to adding the second argument of $2 value to the _command_offset function using the command like sed. but there are external completions as you said. It seems your first option is way to go but that needs to parse the ${COMP_WORDS{COMP_CWORD]} value twice in the script (including quotes, escape sequence). I think it's terrible.

mug896 avatar Aug 25 '22 07:08 mug896

how about using shopt -e extdebug option ? if you enable extdebug option then you can obtain ancestor function's argument count and values like this

#!/bin/bash

_command_offset() {
    echo "${FUNCNAME[0]} function argc: ${BASH_ARGC[0]}"
    echo "${FUNCNAME[0]} function arg1: ${BASH_ARGV[1]}"
    echo "${FUNCNAME[0]} function arg2: ${BASH_ARGV[0]}"
    echo "${FUNCNAME[1]} function argc: ${BASH_ARGC[1]}"
    echo "${FUNCNAME[1]} function arg1: ${BASH_ARGV[4]}"
    echo "${FUNCNAME[1]} function arg2: ${BASH_ARGV[3]}"
    echo "${FUNCNAME[1]} function arg3: ${BASH_ARGV[2]}"
}

_foo() {
    _command_offset 44 55
}

main() {
    _foo 11 22 33
}

shopt -s extdebug
main

#####################################

bash$ ./test.sh
_command_offset function argc: 2
_command_offset function arg1: 44
_command_offset function arg2: 55
_foo function argc: 3
_foo function arg1: 11
_foo function arg2: 22
_foo function arg3: 33

mug896 avatar Aug 25 '22 10:08 mug896

my preferred way is to adding the second argument of $2 value > to the _command_offset function using the command like sed. but there are external completions as you said.

I'm actually currently thinking of the possibility of the hybrid solution of analyzing ${COMP_WORDS{COMP_CWORD]} and passing the original $2: We can add a new option to _command_offset to pass the original $2 (like _command_offset --cur="$2" $offset). When the original $2 is passed, _command_offset can just use the passed value. When the option is not specified, _command_offset analyzes ${COMP_WORDS{COMP_CWORD]} to determine a value. For the completion functions inside the bash-completion project, we can modify the codes to pass the original $2. It is better to modify also external completions so that the original $2 is passed to _command_offset, but they would still work mostly even if they are not modified because $2 is reconstructed from ${COMP_WORDS{COMP_CWORD]}.

It seems your first option is way to go but that needs to parse the ${COMP_WORDS{COMP_CWORD]} value twice in the script (including quotes, escape sequence). I think it's terrible.

I'm currently not sure how hard that is. Does it really require us an authentic shell parser? That is the reason we need to investigate the exact rule used by Bash in order to estimate whether this solution can be useful. If that can be processed by regular expressions like this in most cases, that is fine. It is possibly impossible to perfectly reproduce the original rule, but even in that case, I guess the most realistic cases can be covered by some regular expressions.

how about using shopt -e extdebug option ?

Actually, I also thought about that direction independently, but this direction also seems to have other problems that are hard to solve.

  • First, extdebug doesn't just enable BASH_ARGV and BASH_ARGC, but also changes the semantics of DEBUG / ERR / RETURN traps. Specifically, we can set these traps for each function separately when extdebug is ~~not~~ turned off, but it becomes impossible when extdebug is turned on. This means that if we set extdebug globally, it can break existing configurations that use these traps. In particular, DEBUG trap is used by various Bash configurations like bash-preexec, iTerm2 shell integration, wezterm shell integration, starship, liqiuidprompt, etc.
  • Another concern is the performance. With extdebug, the above traps start to be inherited by all the function calls, so the shell can be significantly slower than without extdebug. Even when no traps are set, I have doubts on the performance of BASH_ARGV and extdebug. Once I have measured the performance of a utility function that uses the hack of BASH_ARGV with extdebug introduced in Pure Bash Bible, but the result was by far inferior to the naive implementation. (By the way, the function of Pure Bash Bible is also broken by globally set extdebug, though it can be easily fixed.)

Then, another possibility that comes to our mind is to locally set extdebug instead of globally setting extdebug. We might think of temporarily turning extdebug on in _command_offset, getting the arguments, and finally unsetting extdebug (or restoring the original state of extdebug). However, this naive solution doesn't work because BASH_ARGV only contains the arguments after we set extdebug. When _command_offset is called, it is too late to set extdebug in order to retrieve the arguments of the caller by BASH_ARGV.

Next, we might consider locally setting extdebug inside the top-level completion function prior to the call of _command_offset. This can be technically possible if the top-level completion function calls _init_completion at the beginning. We can set extdebug when _init_completion is called. At the same time, _init_completion shall set the RETURN trap for the top-level completion function. We can reset extdebug in the RETURN trap. This kind of strategy was experimentally implemented in . PR #739 but is not yet fully tested. This strategy also has some other problems:

  • We cannot ensure that the RETURN trap is called, for example, when the user cancels the completion by C-c (SIGINT). In such a case, we fail to restore the global setting of extdebug. Issue #786 is essentially the same problem. We may think of trapping SIGINT, but we are not sure if that covers all the similar cases.
  • This doesn't always work for external completions. For this to work, _init_completion needs to be called from the top level of the completion function, but _init_completion has never been put such a restriction, so external completions may call it from inside child functions. Also, there seem to be even some external completion functions that do not call _init_completion but only call _command_offset. We can find such completion functions, e.g., here and there.

akinomyoga avatar Aug 25 '22 13:08 akinomyoga

As you already know, quotes are members of the $COMP_WORDBREAKS variable and have different behavior than other WORDBREAKERS. The primary usage of quotes in tab completions is to input multiple words sentence something like this

1) install                              
2) Linked Base for node-1 and node-2     # completion list
3) Linked Base for node-1 and node-3
4) hostname hosts setup

sh$ hello --foo "Linked Ba[tab]
sh$ hello --foo="Linked Ba[tab]
sh$ hello -A "Linked Ba[tab]

and if a user wants to include quotes in the completion list then he first has to escape the quotes like (1) when assign COMPREPLY variable in case (2) will not appear the quotes in the completion list when user enter tab how to tab complete the quotes in the prompt ? you also need to escape the quotes to complete.

install
Linked Base for \"node-1\" and \"node-2\"     ---- (1)
Linked Base for "node-1" and "node-3"         ---- (2)
hostname hosts setup

nobody uses quotes in tab completion like this.

test1 a"b[TAB]
test1 a"b"[TAB]
test1 a"b"c[TAB]
test1 a"b"c"[TAB]

# but this is possible
test1 "ab[TAB]

also nobody uses tab completion on already completed word like this. so these two cases are no need to concern to set the bash's original $2 variable.

test1 a[TAB]bcd
test1 ab[TAB]cd
test1 abc[TAB]d

# but this case is need to concern because following usage pattern
test1 [TAB]abcd    
sh$ hello -a foo -b bar      # user wants to change option argument of -a
sh$ hello -a [TAB]-b bar     # $2: empty    ${COMP_WORDS[COMP_CWORD]}: -b

# because of the current bash-completion bug every completion function 
# that does not use base-completion package has to add this line on top.
# if a user can use the bash's original $2 value then don't need this line
# [[ ${COMP_LINE:COMP_POINT-1:1} = " " ]] && cur=""

So my recommandation is that there is no need to parse ${COMP_WORDS[COMP_CWORD]} variable to set $2 variable. and this is enough to set the original $2 variable.

if [[ ${COMP_WORDS[COMP_CWORD]} == @(\"*|\'*) ]]; then
    set -- "$1" "${COMP_WORDS[COMP_CWORD]:1}" "${COMP_WORDS[COMP_CWORD-1]}"
elif [[ $COMP_WORDBREAKS == *${COMP_WORDS[COMP_CWORD]}* ]] ||
    [[ ${COMP_LINE:COMP_POINT-1:1} = " "  ]]; then
    set -- "$1" "" "${COMP_WORDS[COMP_CWORD-1]}"
else    
    set -- "$1" "${COMP_WORDS[COMP_CWORD]}" "${COMP_WORDS[COMP_CWORD-1]}"
fi

I currently noticed that bash-completion package also set bash's default $3 value to empty then you have to use ${COMP_WORDS[COMP_CWORD-1]} for setting $3 variable

All these are the factors that make people stop trying to learn completion functions and only a few people make it. think about how confusing it ! if the bug I suggested is fixed, then a lot of confusion will be removed..

mug896 avatar Aug 26 '22 03:08 mug896

I created PR #791 that determines $2 based on ${COMP_WORDS[CWORD]}.

I have also implemented what you have suggested, (i.e., letting each completion function pass the original $2) in https://github.com/akinomyoga/bash-completion/commit/3d5869cb5d7eefbc4db673cc88fbe56d38a6878c, but it is not included in the PR currently because the extraction from ${COMP_WORDS[CWORD]} seems to just work fine and the _command_offset interface doesn't seem to need to be changed.

The primary usage of quotes in tab completions is to input multiple words sentence something like this [...]

I am talking about the pattern a"b"c"d"e but not the case like a"b\"c\"d"e. If you call "a\"b\"c" primary usage, I am talking about secondary usage.

nobody uses quotes in tab completion like this. [...]

also nobody uses tab completion on already completed word like this. [...]

I'm wondering how many people that "nobody" is out of. How many people did you actually confirm with that they do not use them? As for the former, I do use like /path/to/"a directory containing space"/sub/directory/. I can easily enumerate more examples:

$ gcc -L"/xxx/yyy/zzz" -L"/xxx/yyy/www" a.c -lfoo -lbar -Wl,-R"/xxx/yyy/zzz",-R"[TAB]
$ echo "result.a=$(xxx).b="*".c=1.txt" or result.a-"$(xxx)".b-*.c-"[TAB]

As for the latter, many people use it as far as their keyboard has the cursor key ⬅️. Consider the reason why Readline has an option skip-completed-text. If nobody would really use it, the option skip-completed-text doesn't have meaning. Anyway, both cases are valid Bash syntax and don't explicitly fail. In general, as far as a feature or usage is not explicitly forbidden by Bash, there is probably someone using it in the world, so we can hardly say nobody about any features.

Even if nobody is using them, I'm wondering if there is a reason to prefer an incomplete ad-hoc solution to the general solution.

akinomyoga avatar Aug 27 '22 03:08 akinomyoga

I made this simple parser for parsing the ${COMP_WORDS[COMP_CWORD]} value to set up the original $2 variable based on your test cases. please check this out whether need to fix or add functionality.

_parse_print() 
{
    echo
    echo '$1: '"[$1]"
    echo '$2: '"[$2]"
    echo '$3: '"[$3]"
}
_parse() 
{
    local comp=${COMP_WORDS[COMP_CWORD]}
    local match=0 two i
    
    if [[ ${comp:0:1} != @(\"|\') && $COMP_WORDBREAKS == *"${comp:0:1}"* ]]; then
        _parse_print "$1" "" "${COMP_WORDS[COMP_CWORD-1]}"
        return
    fi

    for (( i = 0; i < ${#comp}; i++ )); do
        [[ ${COMP_LINE:0:COMP_POINT} == *"${comp:0:i+1}" ]] && let match=i+1
    done

    local str=${comp:0:match}
    local open=-1 curq=""

    for ((i = 0; i < ${#str}; i++)); do
        case ${str:i:1} in
            \") 
                [[ $curq == single ]] && continue
                (( open == -1 )) && { open=$i; curq="double" ;} || { open=-1; curq="" ;}
                ;;
            \') 
                [[ $curq == double ]] && continue
                (( open == -1 )) && { open=$i; curq="single" ;} || { open=-1; curq="" ;}
                ;;
            \\) 
                [[ ${str:i+1:1} == @(\"|\'|\\) && $curq != single ]] && let i++
                ;;
        esac
    done
    if (( open == -1 )); then
        two=$str
    else
        str=${str:open} 
        two=${str#@(\"|\')}
    fi
    _parse_print "$1" "$two" "${COMP_WORDS[COMP_CWORD-1]}"
}
complete -F _parse parse

mug896 avatar Aug 27 '22 12:08 mug896

Thanks, but I actually have already implemented it in #791. My current implementation is just two regex matching and covers all the test cases that I have obtained by Bash 5.2-rc3. Note that I have finally investigated the Bash behavior by myself and added more test cases in addition to the ones in https://github.com/scop/bash-completion/issues/790#issuecomment-1223874518.

The actual rule seems to be more complicated than something that can be guessed from the initial test cases in https://github.com/scop/bash-completion/issues/790#issuecomment-1223874518. In fact, I have tested your implementation with my test cases including new ones, but it only passes 48/126 cases. (The implementation of #791 passes all the cases). Here are the failing cases of your implementation (edit: sorry, I forgot to include the information of COMP_WORDBREAKS for each test case. Now I have added it below):

# default COMP_WORDBREAKS
159: failed: input=<a`echo w> output=<a`echo w> expect=<w>
163: failed: input=<a`bbb ccc`> output=<a`bbb ccc`> expect=<ccc`>
170: failed: input=<$'a b'c`d e> output=<$'a b'c`d e> expect=<e>
171: failed: input=<a`b'c'd e> output=<a`b'c'd e> expect=<e>
172: failed: input=<a`b'c'd e f> output=<a`b'c'd e f> expect=<f>
173: failed: input=<a`$(echo world> output=<a`$(echo world> expect=<world>
175: failed: input=<a`$'b c\'d e$'f g\'> output=<a`$'b c\'d e$'f g\'> expect=<g\'>
176: failed: input=<a`$'b c\'d e$'f g\'h i> output=<a`$'b c\'d e$'f g\'h i> expect=<i>
177: failed: input=<a`$'b c\'d e$'f g\'h i`j> output=<a`$'b c\'d e$'f g\'h i`j> expect=<i`j>
178: failed: input=<a`$'b c\'d e'f g'> output=<a`$'b c\'d e'f g'> expect=<g'>
179: failed: input=<a`a;> output=<a`a;> expect=<>
180: failed: input=<a`x=> output=<a`x=> expect=<>
181: failed: input=<a`x=y> output=<a`x=y> expect=<y>
182: failed: input=<a`b|> output=<a`b|> expect=<>
183: failed: input=<a`b:c> output=<a`b:c> expect=<c>
184: failed: input=<a`b&> output=<a`b&> expect=<>
# COMP_WORDBREAKS=@$IFS
188: failed: input=<a`b@c> output=<a`b@c> expect=<@c>
# COMP_WORDBREAKS=z$IFS
193: failed: input=<a`bzc> output=<a`bzc> expect=<c>
194: failed: input=<a`bzcdze> output=<a`bzcdze> expect=<e>
195: failed: input=<a`bzcdzze> output=<a`bzcdzze> expect=<e>
196: failed: input=<a`bzcdzzze> output=<a`bzcdzzze> expect=<e>
# COMP_WORDBREAKS='$'$IFS
202: failed: input=<a`b$> output=<a`b$> expect=<$>
203: failed: input=<a`b$x> output=<a`b$x> expect=<$x>
204: failed: input=<a`b${> output=<a`b${> expect=<${>
205: failed: input=<a`b${x}> output=<a`b${x}> expect=<${x}>
206: failed: input=<a`b${x}y> output=<a`b${x}y> expect=<${x}y>
207: failed: input=<a`b$=> output=<a`b$=> expect=<$=>
208: failed: input=<a`b$.> output=<a`b$.> expect=<$.>
210: failed: input=<a`b$"a"> output=<a`b$"a"> expect=<$"a">
# COMP_WORDBREAKS='\'$IFS
216: failed: input=<a`b\cd> output=<a`b\cd> expect=<cd>
217: failed: input=<a`b\cde\fg> output=<a`b\cde\fg> expect=<fg>
218: failed: input=<a`b\c\\a> output=<a`b\c\\a> expect=<\a>
219: failed: input=<a`b\c\\\a> output=<a`b\c\\\a> expect=<a>
220: failed: input=<a`b\c\\\\a> output=<a`b\c\\\\a> expect=<\a>
221: failed: input=<a`b\c\a\a> output=<a`b\c\a\a> expect=<a>
222: failed: input=<a`b\> output=<a`b\> expect=<>
223: failed: input=<a`b\\> output=<a`b\\> expect=<\>
224: failed: input=<a`b\\\> output=<a`b\\\> expect=<>
# COMP_WORDBREAKS='\$'$IFS
228: failed: input=<a`b$\> output=<a`b$\> expect=<>
229: failed: input=<a`b\$> output=<a`b\$> expect=<$>
# COMP_WORDBREAKS='$z@'$IFS
233: failed: input=<a$z> output=<a$z> expect=<>
234: failed: input=<a$$z> output=<a$$z> expect=<>
235: failed: input=<a$$> output=<a$$> expect=<$>
236: failed: input=<a$@> output=<a$@> expect=<@>
237: failed: input=<a$$@> output=<a$$@> expect=<@>
# COMP_WORDBREAKS='!'$IFS
240: failed: input=<a`b!> output=<a`b!> expect=<>
241: failed: input=<a`b!c> output=<a`b!c> expect=<c>
# COMP_WORDBREAKS='#'$IFS
243: failed: input=<a`b#> output=<a`b#> expect=<>
244: failed: input=<a`b#c> output=<a`b#c> expect=<c>
# COMP_WORDBREAKS='%'$IFS
246: failed: input=<a`b%> output=<a`b%> expect=<>
247: failed: input=<a`b%c> output=<a`b%c> expect=<c>
# COMP_WORDBREAKS='*'$IFS
249: failed: input=<a`b*> output=<a`b*> expect=<>
250: failed: input=<a`b*c> output=<a`b*c> expect=<c>
# COMP_WORDBREAKS='+'$IFS
252: failed: input=<a`b+> output=<a`b+> expect=<>
253: failed: input=<a`b+c> output=<a`b+c> expect=<c>
# COMP_WORDBREAKS=','$IFS
255: failed: input=<a`b,> output=<a`b,> expect=<>
256: failed: input=<a`b,c> output=<a`b,c> expect=<c>
# COMP_WORDBREAKS='-'$IFS
258: failed: input=<a`b-> output=<a`b-> expect=<>
259: failed: input=<a`b-c> output=<a`b-c> expect=<c>
# COMP_WORDBREAKS='.'$IFS
261: failed: input=<a`b.> output=<a`b.> expect=<>
262: failed: input=<a`b.c> output=<a`b.c> expect=<c>
# COMP_WORDBREAKS='/'$IFS
264: failed: input=<a`b/> output=<a`b/> expect=<>
265: failed: input=<a`b/c> output=<a`b/c> expect=<c>
# COMP_WORDBREAKS='?'$IFS
267: failed: input=<a`b?> output=<a`b?> expect=<>
268: failed: input=<a`b?c> output=<a`b?c> expect=<c>
# COMP_WORDBREAKS='['$IFS
270: failed: input=<a`b[> output=<a`b[> expect=<>
271: failed: input=<a`b[c> output=<a`b[c> expect=<c>
# COMP_WORDBREAKS=']'$IFS
273: failed: input=<a`b]> output=<a`b]> expect=<>
274: failed: input=<a`b]c> output=<a`b]c> expect=<c>
# COMP_WORDBREAKS='^'$IFS
276: failed: input=<a`b^> output=<a`b^> expect=<>
277: failed: input=<a`b^c> output=<a`b^c> expect=<c>
# COMP_WORDBREAKS='_'$IFS
279: failed: input=<a`b_> output=<a`b_> expect=<>
280: failed: input=<a`b_c> output=<a`b_c> expect=<c>
# COMP_WORDBREAKS='}'$IFS
282: failed: input=<a`b}> output=<a`b}> expect=<>
283: failed: input=<a`b}c> output=<a`b}c> expect=<c>
# COMP_WORDBREAKS='~'$IFS
285: failed: input=<a`b~> output=<a`b~> expect=<>
286: failed: input=<a`b~c> output=<a`b~c> expect=<c>
# COMP_WORDBREAKS='`'$IFS
288: failed: input=<a`b`> output=<a`b`> expect=<>

akinomyoga avatar Aug 27 '22 13:08 akinomyoga

I have put the test code that I used in gh790-test.sh. You can download it and run it to perform the tests.

$ ./gh790-test.sh         # test the implementation of PR #791
$ ./gh790-test.sh mug896  # test your implementation

akinomyoga avatar Aug 27 '22 13:08 akinomyoga

I'm currently using bash version 5.1.16 Ubuntu 22.04.1 LTS There seems to be a lot of different values from the expected values. for example

. . .
285: failed: input=<a`b~> output=<a`b~> expect=<>
286: failed: input=<a`b~c> output=<a`b~c> expect=<c>
288: failed: input=<a`b`> output=<a`b`> expect=<>

my simple value printing completion function outputs different than expected value. I also checked on google cloud shell terminal that use bash version 5.1.4(1)-release

. . .
(mine, google)           (expect)
285:  a`b~               expect=<>
286:  a`b~c              expect=<c>
288:  a`b`               expect=<>

mug896 avatar Aug 27 '22 13:08 mug896

Did you set COMP_WORDBREAKS? As far as I try in Bash 5.1.0, 5.1.8, and 5.1.16 in Fedora 36, Bash produces empty results with those cases with the corresponding COMP_WORDBREAKS.

I also tried Bash 5.1.4 in Google Cloud Shell, but the result is the same:

image

akinomyoga avatar Aug 28 '22 00:08 akinomyoga