Issue with File Path Autocompletion in Argparse Outside Virtual Environment

[Alfystar]

Hello,

I’m experiencing an issue with the argparse module in Python, where file path autocompletion is not functioning correctly when not using a virtual environment. The autocompletion feature, which I have enabled with argcomplete, is suggesting other optional parameters instead of file paths.

This problem occurs with a script that utilizes several nested subparsers. I’ve followed the standard procedures to enable autocompletion, including installing argcomplete, using the activate-global-python-argcomplete command, and verifying my shell configuration. Despite these efforts, the expected file path suggestions are not appearing.

Interestingly, activating a virtual environment based on Python 3.12.3 seems to resolve the issue temporarily. However, once I deactivate the virtual environment and revert to the global Python installation, the problem returns.

For a detailed description of the issue, including the structure of the subparsers and the steps I’ve taken to troubleshoot, please refer to my Stack Overflow question: Autocomplete Fails for File Paths in Argparse Without Virtual Environment.

Any insights or assistance with this matter would be greatly appreciated.

Thank you!

[evanunderscore]

Are you able to whittle your example down to the minimum code required to trigger the issue you're seeing? Once you've done that, you should be able to export _ARC_DEBUG=1 to get debug output and see where execution diverges between a working configuration and a broken one.

[Alfystar]

Hi @evanunderscore

This is the minimum code that show the problem:

#!/usr/bin/env python3
# PYTHON_ARGCOMPLETE_OK

import argparse
import argcomplete

def main():
    # Main parser definition
    parser = argparse.ArgumentParser(description="Example of argparse with subcommands.")
    parser.add_argument("--path", type=str, help="A single file path.")
	
	# Enabling autocompletion
    argcomplete.autocomplete(parser)
    # Parsing known arguments
    args, unknown = parser.parse_known_args()

This is my current directory:

12:03:00 ~/Documents/00_My/argvParseComplete
└─▶ $ tree
.
├── Cartella_1
│   ├── File_1.txt
│   ├── File_2.txt
│   └── File_3.txt
├── Cartella_2
│   ├── File_1.txt
│   ├── File_2.txt
│   └── File_3.txt
├── Cartella_3
│   ├── File_1.txt
│   ├── File_2.txt
│   └── File_3.txt
├── Cartella_4
│   ├── File_1.txt
│   ├── File_2.txt
│   └── File_3.txt
├── Cartella_5
│   ├── File_1.txt
│   ├── File_2.txt
│   └── File_3.txt
├── argvParse_example.py
└── createEnvTest.sh

6 directories, 17 files

Output outside virtual Env

And this the output with the export _ARC_DEBUG=1 :

12:05:40 ~/Documents/00_My/argvParseComplete
└─▶ $ ./argvParse_example.py --path  <Type TAB>
word --path split, lexer state: ' '
In trailing whitespace

LINE: './argvParse_example.py --path ' 
POINT: 30 
PREQUOTE: '' 
PREFIX: '' 
SUFFIX: '' 
WORDS: ['./argvParse_example.py', '--path']
Active parsers: [MonkeyPatchedIntrospectiveArgumentParser(prog='argvParse_example.py', usage=None, description='Example of argparse with subcommands.', formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)]
Visited positionals: [MonkeyPatchedIntrospectiveArgumentParser(prog='argvParse_example.py', usage=None, description='Example of argparse with subcommands.', formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)]
invoking parser with ['--path']

exception <class 'ValueError'> too many values to unpack (expected 3) while parsing args
all active parsers: [MonkeyPatchedIntrospectiveArgumentParser(prog='argvParse_example.py', usage=None, description='Example of argparse with subcommands.', formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)]
active_parser: MonkeyPatchedIntrospectiveArgumentParser(prog='argvParse_example.py', usage=None, description='Example of argparse with subcommands.', formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)
optional options: ['-h', '--help', '--path']
next_positional: None
Active actions (L=0): []
active options: ['-h', '--help', '--path']
display completions: {'-h': 'show this help message and exit', '--help': 'show this help message and exit', '--path': 'A single file path.'}

Returning completions: ['-h:show this help message and exit', '--help:show this help message and exit', '--path:A single file path.']
-

Output inside virtual Env

12:07:39 ~/Documents/00_My/argvParseComplete
└─▶ $ source .venv/bin/activate
(.venv) 12:07:47 ~/Documents/00_My/argvParseComplete
└─▶ $ ./argvParse_example.py --path -
word --path split, lexer state: ' '
In trailing whitespace

LINE: './argvParse_example.py --path ' 
POINT: 30 
PREQUOTE: '' 
PREFIX: '' 
SUFFIX: '' 
WORDS: ['./argvParse_example.py', '--path']
Active parsers: [MonkeyPatchedIntrospectiveArgumentParser(prog='argvParse_example.py', usage=None, description='Example of argparse with subcommands.', formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)]
Visited positionals: [MonkeyPatchedIntrospectiveArgumentParser(prog='argvParse_example.py', usage=None, description='Example of argparse with subcommands.', formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)]
invoking parser with ['--path']

exception <class 'SystemExit'> 2 while parsing args
all active parsers: [MonkeyPatchedIntrospectiveArgumentParser(prog='argvParse_example.py', usage=None, description='Example of argparse with subcommands.', formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)]
active_parser: MonkeyPatchedIntrospectiveArgumentParser(prog='argvParse_example.py', usage=None, description='Example of argparse with subcommands.', formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)
optional options: ['-h', '--help', '--path']
next_positional: None
Active actions (L=1): [IntrospectAction(option_strings=['--path'], dest='path', nargs=None, const=None, default=None, type=<class 'str'>, choices=None, required=True, help='A single file path.', metavar=None)]
Resetting completions because IntrospectAction(option_strings=['--path'], dest='path', nargs=None, const=None, default=None, type=<class 'str'>, choices=None, required=True, help='A single file path.', metavar=None) must consume the next argument
Activating completion for IntrospectAction(option_strings=['--path'], dest='path', nargs=None, const=None, default=None, type=<class 'str'>, choices=None, required=True, help='A single file path.', metavar=None) <class 'argparse._StoreAction'>
Completions: ['createEnvTest.sh', 'argvParse_example.py', 'Cartella_1/', 'Cartella_2/', 'Cartella_5/', '.venv/', 'Cartella_4/', 'Cartella_3/', '.idea/']
active options: ['createEnvTest.sh', 'argvParse_example.py', 'Cartella_1/', 'Cartella_2/', 'Cartella_5/', '.venv/', 'Cartella_4/', 'Cartella_3/', '.idea/']
display completions: {'createEnvTest.sh': '', 'argvParse_example.py': '', 'Cartella_1/': '', 'Cartella_2/': '', 'Cartella_5/': '', '.venv/': '', 'Cartella_4/': '', 'Cartella_3/': '', '.idea/': ''}

Returning completions: ['createEnvTest.sh:', 'argvParse_example.py:', 'Cartella_1/:', 'Cartella_2/:', 'Cartella_5/:', '.venv/:', 'Cartella_4/:', 'Cartella_3/:', '.idea/:']
      ./argvParse_example.py --path  
usage: argvParse_example.py [-h] --path PATH
argvParse_example.py: error: argument --path: expected one argumentartella_1/  .venv/  --    

Hope this data can help you! I'm here if you need

[evanunderscore]

Which versions of Python are you using inside and outside the virtual environment? At a glance this appears to be related to #481.

[Alfystar]

Boot (env and global) are with Python 3.12.3 And i'm on Mac using zsh and bash, if can help

[Alfystar]

Any news?

[evanunderscore]

I'm sorry, I haven't had time to look into this. Something appears to be going wrong here:

exception <class 'ValueError'> too many values to unpack (expected 3) while parsing args

I would ordinarily expect to see SystemExit as you do inside the virtual environment. If you're able to alter the argparse code to show the traceback, that might help us work out what the issue is.

[FlorianPommerening]

From the error message and your Python version, it sounds like the incompatibility older versions had with Python 3.12.3+. This was fixed in 54a4846 and released in version 3.3. What argcomplete version are you using?

[Alfystar]

Sorry for the delay, I was on my honeymoon anyway, I just saw and installed 3.2.2 With Python 3.12.4

Now update to 3.5.0 and finally work good 😄

THANK YOU!!! ❤️