Click-repl pkg tab-completion of option values fails - need new click-repl version with fix

[KSchopmeyer]

Click-repl package version 0.2.0 tab-completion of option values (i.e. in the repl mode) fails. The package now has a fix for this issue https://github.com/click-contrib/click-repl/pull/88 which has been committed to the master but it has not been released. For us this means that the tab-completion for options like --name, --mock-server, --connection-file, etc. will not work in repl mode but will work in command line mode.

However, the changes for this new release (date unknown) also removes support for python 2.

The alternatives are probably to:

1. localize that package based on its current master until they do a release (ex. fork it). I recommend we not do this.
2. Document the issue and note to users that the repl tab-completion of these option/argument values does not work in this release.

Proposal: document the deficiency in pywbemcli docs but not try to use their master branch as our package version.

Issue noted in documentation for tab-completion in pr #1282, (generalization of tab-completion documentation) as a note.

Note that there is a second issue with click-repl in that the tab completion does not include tab-completion for the click.Path() type which is part of tab-completion for all of the shells which click-tab-completion supports. This is documented in click-repl issues in issue https://github.com/click-contrib/click-repl/issues/59 but since that issue is several years old, it would appear that nobody is taking responsibility for a fix.

[andy-maier]

Do the repl users really use tab completion on options that change the target server?

I think that we should use the released version.

Karl created a new Issue #1441 to define the questions, etc. about using click-repl v2 vs v3.0 for the pywbemtools 1.4.0 release since this issue has gotten very large with a set of problems and discussions around the work being done by GhostOps in summer 2023 and in the mean time a version 3.0 click_repl was release that are at least partly resolved but not completely resloved these issues.

DISCUSSION: I propose we close this issue as superceeded

[KSchopmeyer]

We concluded that we will note this restriction in the docs and the changes.rst.

[KSchopmeyer]

The note about the restriction and issues with python 2.7 and python 3.5 was included in PR #1247 and PR #1278. I proposed that this issue be held open but not tied to pywbemtools 1.2.0 as it is dependent on the release of the next version of click-repl.

DISCUSSION

[GhostOps77]

So, do u want the python 2 support for the click-repl package?

[KSchopmeyer]

Thanks for asking the question above.

We still support python 2/3.5 but since the whole premise of tab-complete/autocomplete is different between click 8 and click 7 and we think that the primary users for python 2 are some older autoated test environments, not our interactive users, we are more comfortable with python 2 not including the completion of argument and option values. Therefore since the current click-repl version is python 3 only, we define the old click-repl version for python 2 and python 3.5 as part of install requirements. We are probably better off not worrying about tab-completion for args and options with click 7 because that would lead to other differences between the shell tab-completion and repl tab completion (the click at option/argument shell_completion, etc.

Actually what we really would like is the release of a new click-repl version that includes click-repl pr https://github.com/click-contrib/click-repl/pull/88 (the extension to handle the argument and option values) in the repl mode ASAP. Right now we put a note in our documentation that argument/option value tab-completion does not work in repl mode for this release. Note that since we have not set a maximum click-repl version in requirements, if a new release is made, the args/option value tab-completion in our code should work (sort of a surprise for the user).

Anything I can to to speed up the next click-repl release I will be glad to help with.

NOTE: I did run a test of the click-repl master in our pywbemtools test suite and also manually in repl mode and I think I hit a failure with processing click.path options which works in the shell but not in the interactive mode( see below). I have not filed a bug because I want to create a simple example to demo the issue. NOTE: We do not have functional tests for tab-completion from the command line or in interactive mode in our test suite yet. We just have unit tests of our own shell completer functions so the pywbemtools automated tests do not pick this up.

Example of the issue I a seeing now:

  With an option that has the type click.Path() and no shell_completion (i.e, pywbemcli --connections-file)

   In the bash shell:

    pywbemcli -C /home/kschopmeyer/.pywbemcli_connect<TAB>
    produces:
    pywbemcli -C /home/kschopmeyer/.pywbemcli_connections.yaml

In the repl mode (with the clic-repl master branch):
    pywbmcli$  -C /home/kschopmeyer/.pywbemcli_connect<TAB>
   does nothing

at the same time:
With an option that is type str and includes the shell_complete attribute:

in both the repl mode and command mode, the tab-completion works:
   
   pywbemcli$  -n mock3n<TAB>
   produces:
                       -n mock3ns        <----which is correct.

[GhostOps77]

Thanks for asking the question above.

We still support python 2/3.5 but since the whole premise of tab-complete/autocomplete is different between click 8 and click 7 and we think that the primary users for python 2 are some older autoated test environments, not our interactive users, we are more comfortable with python 2 not including the completion of argument and option values. Therefore since the current click-repl version is python 3 only, we define the old click-repl version for python 2 and python 3.5 as part of install requirements. We are probably better off not worrying about tab-completion for args and options with click 7 because that would lead to other differences between the shell tab-completion and repl tab completion (the click at option/argument shell_completion, etc.

Actually what we really would like is the release of a new click-repl version that includes click-repl pr click-contrib/click-repl#88 (the extension to handle the argument and option values) in the repl mode ASAP. Right now we put a note in our documentation that argument/option value tab-completion does not work in repl mode for this release. Note that since we have not set a maximum click-repl version in requirements, if a new release is made, the args/option value tab-completion in our code should work (sort of a surprise for the user).

Anything I can to to speed up the next click-repl release I will be glad to help with.

NOTE: I did run a test of the click-repl master in our pywbemtools test suite and also manually in repl mode and I think I hit a failure with processing click.path options which works in the shell but not in the interactive mode( see below). I have not filed a bug because I want to create a simple example to demo the issue. NOTE: We do not have functional tests for tab-completion from the command line or in interactive mode in our test suite yet. We just have unit tests of our own shell completer functions so the pywbemtools automated tests do not pick this up.

Example of the issue I a seeing now:

  With an option that has the type click.Path() and no shell_completion (i.e, pywbemcli --connections-file)

   In the bash shell:

    pywbemcli -C /home/kschopmeyer/.pywbemcli_connect<TAB>
    produces:
    pywbemcli -C /home/kschopmeyer/.pywbemcli_connections.yaml

In the repl mode (with the clic-repl master branch):
    pywbmcli$  -C /home/kschopmeyer/.pywbemcli_connect<TAB>
   does nothing

at the same time:
With an option that is type str and includes the shell_complete attribute:

in both the repl mode and command mode, the tab-completion works:
   
   pywbemcli$  -n mock3n<TAB>
   produces:
                       -n mock3ns        <----which is correct.

And yeah, I understand it why, cuz we still haven't implemented the autocompletion for filenames and filepath

And I also completely apologize, because in one of my PR's, i've made the module to support only for python>=3.7, It'll be reverted back to support python 2 later

Will implement it soon, and ping u later.

And also, though I'm not the maintainer, It'll be a great pleasure if you contribute to the click-repl project from your side as well

Thanks for the reply

------------- Comment ks to GhostOps77

No commitments yet but I suggest we work together on this (i.e. whoever starts the change warn the other since do not want to duplicate.). We are going to release with the current code base and work on issues like this after the release. I actually have several more issues with tab-completion before it is complete but want to get this work out.

There is already an issue open in click-repl ( # 88 )

[KSchopmeyer]

ACTION: What we will need to do in the future is to require the new version of click-repl when it is released.

Karl needs to write an issue in click-repl documenting in simple example the problem with tab-completion master today.

Finally, our documentation today states that the argument value and option value tab-completion does not work, consitent with the current version of click-repl. We need to update the documentation when the next version of click-repl is released.

‘issue already exist in clic-repl for tab-completion problem. See their issue #88.

Not sure about doing issue requesting a release however. Suggestions welcome.

DISCUSSION. This is not a ver 1.2.0 problem any more since the prime concerns are the next version of click-Ripken. Suggest we remove the 1.2.0 milestone.

[GhostOps77]

@KSchopmeyer I'm sorry, but I was busy in my personal life for the past one week that's why I was unable to resume the feature updates in click-repl

I'll start to add those changes to the click-repl from now on Currently, I'm updating all the changes in my click-repl fork (pls ignore the sphagetti code, will get better before its release)

And also as I said, It would be a great if you were able to contribute changes to the click-repl module, even if i'm unable to update the features

Make a new issue about this in the main click-repl repo

Also, I'm having a very big trouble on making the module as python 2 compatible, and I'm unable to run tests for py2 in gh-actions Can you give me a reason that why you are making your module to support for python 2? And if you still want to have the module to support python 2, then you can make changes in the forked repo that I have mentioned above, and make test cases for it as well

For anymore further discussions, you can contact me on discord or any other platform that you are comfortable with

[andy-maier]

MacOS no longer ships Python 2.7 as of its version 12.3, and major Linux distributions have removed it. However, googling "install python 2.7 on linux" has 129 million hits, which tells you something.

Github Actions supports running Python 2.7 even on ubuntu-latest which by now is 22.04 which no longer includes it by default, so the Github folks reinstall it even there.

So we have to realize that people are using Python 2.7 longer than anticipated.

In addition, the pywbem package is included as a package in RHEL. RHEL is pretty much the most conservative Linux distro in terms of supporting old versions of stuff. RedHat supports Python 2.7 in RHEL 8 until 6/2024, and has removed it from RHEL 9. Unfortunately, the RedHat package maintainers of pywbem did not upgrade to our latest version due to new dependencies (nocasedict and nocaselist). In the pywbemtools project I think we would always support the same Python versions as pywbem does, for consistency.

The question is at which point do we drop support for Python 2.7. Maybe we should set 6/2024 as a date for that.

[GhostOps77]

@andy-maier The drop for Python 2 support is up to you and to @KSchopmeyer So, If you guys still want the Py2 support, do you guys mind in supporting in improving the code in this fork? (I can't make the code work for python 2, I really need some help, you can see it in the gh-action workflows)

But my question is, If you guys are going to do the updates in click-repl, where can I contact you guys?

[andy-maier]

Hi @GhostOps77 . Thanks for the offer. We'll discuss it in our weekly meeting tomorrow. We have a public Slack channel for pywbem development at https://pywbem.slack.com/archives/CA2516Z0Q . You're welcome to join there (I think you can just go there - if that does not work, let me know here).

[andy-maier]

I invited you just now.

[andy-maier]

BTW, Karl is travelling this week, so we canceled our weekly meeting this week.

[KSchopmeyer]

Status of this issue:

1. The current release of click-repl does not effectively support tab completion so that use of tab completion in the interactive mode does not really work.
2. Code is complete for most of the required changes but we do not know what the release date is for the next version of click-repl and do not want to wait for that release.
3. There is still one feature missing from click-repl (tab-completion for directories and files) and that is being worked on by another contributor.

Thus we propose to release pywbemtools with the current (0/2.0) version of click-repl and the statement in the documentation that the tab-completion is incomplete. We will rerelease upon release of an updated version of click-repl.

[andy-maier]

The remaining work for us is to require the new click-repl version once it is released. We keep this issue in 1.3.0 for the time being, to see whether a new click-repl version is released by the time pywbemtools 1.3.0 is ready for release.

[KSchopmeyer]

ACTIONS:

1. Clarify what versions of the click-repl code we can use
2. clarify what limitations are in the repl mode.

[KSchopmeyer]

Move this issue to version 1.4.0

[KSchopmeyer]

click-repl 3.0 has been released however, the functionality to make the general options available to the repl command line has disappeared.

However, testing with click-repl 3.0 causes over 80 test failures in the unit pywbemcli tests because we use the general options as part of a number of our tests to test complex functionality that involves multiple command calls to test.

Note that in another PR we have proposed to use the repl more to reduce pywbemtools run time.