Dyanmic shell completion

[JuneStepp]

Is there a way to have parts of an app use dynamic completion? I've been looking to switch from Typer, but have run into the issue that some of my parameters have partially non-static completions that rely on state (config files, ect).

[BrianPugh]

Currently, Cyclopts compiles everything statically into a completion script for performance. However, cyclopts run does perform dynamic completion. It does this through the hidden "_complete" command that the shell-completion script invokes.

With all that said, we don't have a ready made solution, but we can certainly come up with a plan and add it!

[JuneStepp]

Something like this would make sense to me:

• By default, behavior is the same as now.
• Parameters can have a completion function specified that would take the currently typed characters (for this parameter) and return the options to show. (their name and description). The completion script would call Python for these parameters but still be static for everything else.
• Cyclopts exports a global boolean specifying if the code is running as part of shell completion.
• For projects that need more dynanisim like changing available commands or help text, there could be an option to disable static completion, behaving like cyclopts run. This wouldn't exclude the use of completion functions.

[BrianPugh]

I definitely intended something like this when adding Completion support, but it was axed for the sake of getting something out. I think we can revisit the topic now, though.

Let me try and expand your post:

1. Introduce Parameter.completion: Callable | None = None where None means the current behavior (static completion). The handler signature would probably be similar to Click's. I'm not exactly an expert on shell completion, so I'd have to do some research to see if we can simplify the API a bit.

2. Secretly add a hidden command like __cyclopts_completion. Shell-completion scripts would call this if they need to execute python code to generate the completion responses (i.e. when Parameter.completion is not None for a particular argument. An alternative is setting an bool EnvVar as you mentioned (that Cyclopts then checks).

3. there could be an option to disable static completion

   I think this could naturally be exposed via App.default_parameter = Parameter(completion=...). We would need to make sure that the completion handler has enough context to handle it as generically as this.

I'd love to discuss this more! Priority-wise, this is my current todo list for cyclopts:

1. A bunch of minor features/bug-fixes for an upcoming release.
2. Work more on docs generation.
3. This

If you could help me fully flesh out a spec/design-doc for this, it would be much appreciated!

[JuneStepp]

Introduce Parameter.completion: Callable | None = None where None means the current behavior (static completion). The handler signature would probably be similar to Click's. I'm not exactly an expert on shell completion, so I'd have to do some research to see if we can simplify the API a bit.

Potential example of that:

Parameter.completion: Callable[[str], Sequence[str | tuple[str, str]]] | None

def complete_pi(incomplete: str) -> Sequence[str | tuple[str, str]]]
    return ("3", "1", ("4", "standard precision"), "5", "9")

It may be that everything besides the completion and its description can be handled by types. If not, then I think Sequence[str, CompletionItem] would make more sense.

Arguments should be in the order of the sequence, so in fish, --keep-order would need to be used. I haven't investigated other shells.

Problem that I've had that leads to a concern with this: One of the problems I've had is the lack of context i.e. dealing with validations and commands that depend on other commands/parameters. For commands, layers of meta apps can be used if one deals with some of the issues they bring*, but validation and conversion functions have no access to any extra non-parsed parameters from a meta app, forcing the use of global variables or nested functions with nonlocal variables (classes don't improve this). One sort of solution could be doing the validation in the command and manually creating the error messages, but that would still leave these completion functions with no access. Is there something I'm missing here?

*parameter order, broken commands (#680), worse typing, inherent complexity.

Secretly add a hidden command like __cyclopts_completion. Shell-completion scripts would call this if they need to execute python code to generate the completion responses (i.e. when Parameter.completion is not None for a particular argument. An alternative is setting an bool EnvVar as you mentioned (that Cyclopts then checks).

I agree a hidden command makes since for that, but I was talking about a boolean the user code can access to know when it is running as part of completions. For example, I may want to avoid raising errors or printing extra information during completion.

there could be an option to disable static completion

I think this could naturally be exposed via App.default_parameter = Parameter(completion=...). We would need to make sure that the completion handler has enough context to handle it as generically as this.

What if someone wanted what commands or options are available to be dynamic?