Using environment variables with FilesCompleter

[FlorianPommerening]

When completing a filename that includes an environment variable with the FilesCompleter, the forwarding to bash doesn't work.

To reproduce

Test environment:

• Ubuntu 24.04
• Python 3.12.3
• argcomplete 3.1.4-1ubuntu0.1 (this is 3.1.4 with a patch back-ported from 3.3.0 to make it compatible with Python 12.3)
• zsh 5.9

I tested this with the following file foo.py

#!/usr/bin/env python3
# PYTHON_ARGCOMPLETE_OK

import argcomplete, argparse

parser = argparse.ArgumentParser()
arg = parser.add_argument("file")
arg.completer = argcomplete.completers.FilesCompleter()

argcomplete.autocomplete(parser)

mkdir files
touch files/aa files/ab
export MY_FILES=$(pwd)/files
./foo.py files/<TAB>     # This suggests ['files/aa', 'files/ab'] and completes to 'files/a' as expected.
./foo.py $MY_FILES/<TAB> # This does not offer any suggestions.

Additional Testing in Python

I believe the cause has something to do with the way that FilesCompleter forwards the call to bash https://github.com/kislyuk/argcomplete/blob/develop/argcomplete/completers.py#L67

 _call(["bash", "-c", "compgen -A file -- '{p}'".format(p=prefix)])

I tried to isolate this like this (same setup as above):

#!/usr/bin/env python3

import subprocess

cmd = ["bash", "-c", "compgen -A file -- '$MY_FILES/'"]
out = subprocess.check_call(cmd, universal_newlines=True)
print(out)

Executing this script fails both in zsh and in bash. The test above only failed in zsh, not in bash but I cannot see how it would succeed in bash if calling the check_call fails. Maybe I only see the correct completions in bash because the command fails and then falls back to the default completion in bash?

Additional Testing in zsh/bash

bash -c "compgen -A file -- '$MY_FILES/'"

Resolves $MY_FILES before the call and effectively calls

bash -c "compgen -A file -- '/path/to/files/'"

and returns

/path/to/files/ab
/path/to/files/aa

This works both in bash and zsh but is not what the FilesCompleter calls and not the completion we expect. We have to escape the $.

bash -c "compgen -A file -- '\$MY_FILES/'"

This doesn't print anything and returns with exit code 1 (both bash and zsh). However, starting a new bash, and then executing compgen -A file -- '\$MY_FILES/' correctly outputs

$MY_FILES/ab
$MY_FILES/aa

I don't understand why bash -c <command> would behave differently from opening a new bash and executing <command> there. I also tried with -i and -l to make the shell interactive, or a login shell with no success. bash -c env also confirmed that $MY_FILES is available in the nested shell.