Fail to load all the helps by python code when 'arcdata' installed

[daxianji007]

Related command

not sure

Extension name (the extension in question)

arcdata

Description of issue (in as much detail as possible)

When I try to get all the helps by python code when 'arcdata' installed. There is exception. Check my scripts:

pip install azure-cli
az extension add -n arcdata

and python code:

from azure.cli.core import MainCommandsLoader, AzCli
from azure.cli.core.commands import AzCliCommandInvoker, ExtensionCommandSource
from azure.cli.core.parser import AzCliCommandParser
from azure.cli.core.file_util import create_invoker_and_load_cmds_and_args, get_all_help
from azure.cli.core._help import AzCliHelp, CliCommandHelpFile
from unittest.mock import patch

az_cli = AzCli(cli_name='az',
        commands_loader_cls=MainCommandsLoader,
        invocation_cls=AzCliCommandInvoker,
        parser_cls=AzCliCommandParser,
        help_cls=AzCliHelp)

with patch('getpass.getuser', return_value='your_system_user_login_name'):
    create_invoker_and_load_cmds_and_args(az_cli)

help_files = get_all_help(az_cli)

exception is:

Exception has occurred: ModuleNotFoundError
No module named 'kubernetes.client.exceptions'
  File "C:\Users\shujia\.azure\cliextensions\arcdata\azext_arcdata\core\kubernetes.py", line 21, in <module>
    from kubernetes.client.exceptions import ApiException
  File "C:\Users\shujia\.azure\cliextensions\arcdata\azext_arcdata\sqlmi\custom.py", line 16, in <module>
    import azext_arcdata.core.kubernetes as kubernetes_util
  File "C:\_repos\ReferenceAutomation\AzureCli\ExportCommands.py", line 83, in <module>
    create_invoker_and_load_cmds_and_args(az_cli)

Additional context

I am using this information to generate the Docs Page(https://docs.microsoft.com/en-us/cli/azure/arcdata?view=azure-cli-latest) . This exception could break our pipeline. To workaround this. Currently I install another package 'kubernetes' then the exception disappear.

[yonzhan]

route to CXP team

[ghost]

Thank you for your feedback. This has been routed to the support team for assistance.

[swells]

daxianji007

If you run --debug when installing the extension do you get pip install errors? az extension add -n arcdata --debug

kubernetes is a requirement already?

"name": "arcdata",
                    "run_requires": [
                        {
                            "requires": [
                                "colorama (==0.4.4)",
                                "jinja2 (==3.0.3)",
                                "jsonpatch (==1.24)",
                                "jsonpath-ng (==1.4.3)",
                                "jsonschema (==3.2.0)",
                                "kubernetes (==12.0.1)",
                                "ndjson (==0.3.1)",
                                "pem (==21.2.0)",
                                "pydash (==4.8.0)"
                            ]
                        }

[navba-MSFT]

@daxianji007 Thanks for reaching out to us and reporting this issue. I am looking into this issue. I will update this thread once I have more details on this.

[navba-MSFT]

@daxianji007 Could you please run the command pip3 install kubernetes and then check if the above command works ?

If it doesn't help, please remove the arcdata extension and then try adding the extension again and see if that helps:

az extension remove --name arcdata az extension add --name arcdata

If you still see the same issue, please run the command again with --debug switch and share the output here for further investigation. Awaiting your reply.

[daxianji007]

@navba-MSFT @swells If I install kubernetes manually, this issue is gone. I think I get the root cause now. I have installed both arcappliance and arcdata. Then their dependency package folders are appended to the sys.path in AzureCLI. When arcdata tries to load kubernetes, it finds the one in arcappliance folder, which is in version 11.0.0. Then the importing failes.

[navba-MSFT]

@daxianji007 Thanks for getting, Just to confirm may I know if the actual issue is addressed now ? Also let me know if you have any pending questions.

[daxianji007]

@navba-MSFT em... not exactly addressed. I hope the two extensions can be compatible instead of I manually install some package to work around it.

[navba-MSFT]

Thanks @daxianji007 for getting back and clarifying the ask. I am involving the Service Team to look into this feature ask.