ansible-navigator icon indicating copy to clipboard operation
ansible-navigator copied to clipboard

ansible-navigator not able to list roles in a collection

Open patrick-tang-redhat opened this issue 3 years ago • 25 comments
trafficstars

ISSUE TYPE
  • Bug Report
SUMMARY

While we can list roles and check the documents for roles from automation hub, we cannot list roles with cli ansible-navigator ('ansible-navigator collections').

ANSIBLE-NAVIGATOR VERSION
ansible-navigator 1.1.0
CONFIGURATION
LOG FILE
STEPS TO REPRODUCE
  1. ansible-navigator collections
  2. select a collection with roles, for example 'redhat.insights'. type the number to enter content list of the collection.
  3. check if all content of the selected collection shows roles.
EXPECTED RESULTS

expected: the roles in a collection should be shown.

ACTUAL RESULTS

actual: roles are missing from the content of the collection.

ADDITIONAL INFORMATION

As a comparison, please check following link for example collection to show roles in its content: https://console.redhat.com/ansible/automation-hub/repo/published/redhat/insights/content?showing=all automation hub shows roles ansible-navigator not showing roles

patrick-tang-redhat avatar Apr 06 '22 18:04 patrick-tang-redhat

@patrick-tang-redhat Thanks for this!

Role browsing support was added here: https://github.com/ansible/ansible-navigator/pull/1059 and will be in the 2.0.0 release of ansible-navigator. The upstream release candidate should be out today and the upstream final 2.0.0 next week.

When the RC is out, I'll post a link here so you can give it a run for us and let us know what you think

cidrblock avatar Apr 07 '22 11:04 cidrblock

Thanks @cidrblock Good to know this is duplicated. Looking forward the release. It is not very convenient that not able to get information of roles in cli.

patrick-tang-redhat avatar Apr 07 '22 13:04 patrick-tang-redhat

@cidrblock I tried to run this pre-release version 'ansible-navigator-2.0.0b0' in rhel8, with python39, but got error (i installed with 'pip install --user'): FileNotFoundError: [Errno 2] No such file or directory: '/root/.local/share/ansible_navigator/utils/key_value_store.py'

Do you know the fix?

patrick-tang-redhat avatar Apr 07 '22 17:04 patrick-tang-redhat

@patrick-tang-redhat I see root there, can you try and install it with the --user flag? (if you didn't)

pip install ansible-navigator==2.0.0b0 --user

Still working to get the release candidate out today, keep me posted please

cidrblock avatar Apr 07 '22 17:04 cidrblock

Sorry, I see you did, let me look

cidrblock avatar Apr 07 '22 17:04 cidrblock

Can you run it with:

ansible-navigator --ll debug --la false and send the resulting log file over?

cidrblock avatar Apr 07 '22 17:04 cidrblock

@cidrblock please find below the logs: ` [root@rhel8 share]# ansible-navigator --ll debug --la false

Traceback (most recent call last): File "/bin/ansible-navigator", line 11, in load_entry_point('ansible-navigator==1.1.0', 'console_scripts', 'ansible-navigator')() File "/usr/lib/python3.8/site-packages/ansible_navigator/cli.py", line 102, in main new_messages, new_exit_messages = parse_and_update(sys.argv[1:], args=args, initial=True) File "/usr/lib/python3.8/site-packages/ansible_navigator/initialization.py", line 194, in parse_and_update new_messages, new_exit_messages, cache = get_and_check_collection_doc_cache( File "/usr/lib/python3.8/site-packages/ansible_navigator/initialization.py", line 113, in get_and_check_collection_doc_cache collection_cache = _get_kvs(share_directory, collection_doc_cache_path) File "/usr/lib/python3.8/site-packages/ansible_navigator/initialization.py", line 139, in _get_kvs spec.loader.exec_module(mod) File "", line 779, in exec_module File "", line 915, in get_code File "", line 972, in get_data FileNotFoundError: [Errno 2] No such file or directory: '/root/.local/share/ansible_navigator/utils/key_value_store.py' `

patrick-tang-redhat avatar Apr 07 '22 17:04 patrick-tang-redhat

@cidrblock seems the key_value_store.py is not in the place: ` [root@rhel8 share]# ls -l total 0 drwxr-xr-x. 5 root root 48 Apr 7 13:30 ansible_navigator -rw-r--r--. 1 root root 0 Apr 7 13:31 ansible-navigator.log drwxr-xr-x. 3 root root 19 Apr 7 11:35 ansible-runner [root@rhel8 share]# ls ansible_navigator/ grammar themes utils [root@rhel8 share]# ls ansible_navigator/utils catalog_collections.py image_introspect.py init.py pycache

`

After I manually copied the file 'key_value_store.py' to ~/.local/share/ansible_navigator/utils folder. I'm able to launch ansible-navigator command. However when I tried to check information of one collection, the application crashed with following error: ` Traceback (most recent call last): File "/bin/ansible-navigator", line 11, in load_entry_point('ansible-navigator==1.1.0', 'console_scripts', 'ansible-navigator')() File "/usr/lib/python3.8/site-packages/ansible_navigator/cli.py", line 141, in main return_code = run(args) File "/usr/lib/python3.8/site-packages/ansible_navigator/cli.py", line 82, in run wrapper(ActionRunner(args=args).run) File "/usr/lib64/python3.8/curses/init.py", line 105, in wrapper return func(stdscr, *args, **kwds) File "/usr/lib/python3.8/site-packages/ansible_navigator/action_runner.py", line 64, in run self._run_app(interaction) File "/usr/lib/python3.8/site-packages/ansible_navigator/action_runner.py", line 73, in _run_app interaction = run_action(self.steps.current.name, self.app, self.steps.current) File "/usr/lib/python3.8/site-packages/ansible_navigator/actions/_actions.py", line 97, in run_interactive return action_cls(app.args).run(app=app, interaction=interaction) File "/usr/lib/python3.8/site-packages/ansible_navigator/actions/collections.py", line 138, in run self._take_step() File "/usr/lib/python3.8/site-packages/ansible_navigator/actions/collections.py", line 153, in _take_step result = run_action(self.steps.current.name, self.app, self.steps.current) File "/usr/lib/python3.8/site-packages/ansible_navigator/actions/_actions.py", line 97, in run_interactive return action_cls(app.args).run(app=app, interaction=interaction) File "/usr/lib/python3.8/site-packages/ansible_navigator/actions/select.py", line 37, in run app.steps.append(app.steps.current.select_func()) # add next File "/usr/lib/python3.8/site-packages/ansible_navigator/actions/collections.py", line 199, in _build_plugin_menu for plugin_chksum, details in selected_collection["plugin_chksums"].items(): KeyError: 'plugin_chksums'

`

patrick-tang-redhat avatar Apr 07 '22 17:04 patrick-tang-redhat

Somehow I think your installation is odd, your first message has a mention of _get_kvs which is not in the current code base, and also mentions load_entry_point('ansible-navigator==1.1.0', 'console_scripts', 'ansible-navigator') which is the old version

cidrblock avatar Apr 07 '22 17:04 cidrblock

Can you try with an install in a virtual environment? I can provide the steps, this might be related to an RPM install + a pip installation?

Or uninstall the pip version, uninstall the RPM and then install the pip version

cidrblock avatar Apr 07 '22 17:04 cidrblock

@cidrblock could you provide the steps for virtual environment? I'm not familiar with it. In the meantime, let me clean up the environment to put back.

I installed python39 as module, so everything goes together

patrick-tang-redhat avatar Apr 07 '22 17:04 patrick-tang-redhat

$ mkdir navigator-test
$ cd navigator-test 
$ python3.9 -m venv venv 
$ source venv/bin/activate
$ pip install ansible-navigator==2.0.0b0
$ ansible-navigator --version
ansible-navigator 2.0.0b0

This will build a virtual environment and install navigator in it, when you are done, you can delete the directory and nothing will be left over, clean up is easy. Good for testing

cidrblock avatar Apr 07 '22 17:04 cidrblock

@cidrblock it might have something to do with the ee environment. I have tried to use venv, and 2.0.0b0 worked for me. However, I noticed there is a big difference: with venv, it tried to pull down a new ee image: quay.io/ansible/creator-ee:v0.4.0. While i was using non virtual env, it was using: registry.redhat.io/ansible-automation-platform-21/ee-supported-rhel8:latest

The old error log: ` [root@rhel8 tmp]# ansible-navigator collections

Execution environment image and pull policy overview

Execution environment image name: registry.redhat.io/ansible-automation-platform-21/ee-supported-rhel8:latest Execution environment image tag: latest Execution environment pull policy: tag Execution environment pull needed: True

Updating the execution environment

Trying to pull registry.redhat.io/ansible-automation-platform-21/ee-supported-rhel8:latest... Getting image source signatures Checking if image destination supports signatures Copying blob 510abfcdf6bc skipped: already exists
Copying blob 4aa6faa2acb0 skipped: already exists
Copying blob 916e0bda4eea skipped: already exists
Copying blob 4a3604715398 skipped: already exists
Copying blob 6ccfcef17d4b skipped: already exists
Copying config 43cfa4a7f9 done
Writing manifest to image destination Storing signatures 43cfa4a7f9f7176c797dfc05a211cc754e0cb29e1ffd0d2a1ad954a8dd690b81 Traceback (most recent call last): File "/bin/ansible-navigator", line 11, in load_entry_point('ansible-navigator==1.1.0', 'console_scripts', 'ansible-navigator')() File "/usr/lib/python3.8/site-packages/ansible_navigator/cli.py", line 141, in main return_code = run(args) File "/usr/lib/python3.8/site-packages/ansible_navigator/cli.py", line 82, in run wrapper(ActionRunner(args=args).run) File "/usr/lib64/python3.8/curses/init.py", line 105, in wrapper return func(stdscr, *args, **kwds) File "/usr/lib/python3.8/site-packages/ansible_navigator/action_runner.py", line 58, in run self.initialize_ui(DEFAULT_REFRESH) File "/usr/lib/python3.8/site-packages/ansible_navigator/action_runner.py", line 48, in initialize_ui self._ui = UserInterface( File "/usr/lib/python3.8/site-packages/ansible_navigator/ui_framework/ui.py", line 123, in init super().init(ui_config=ui_config) File "/usr/lib/python3.8/site-packages/ansible_navigator/ui_framework/curses_window.py", line 63, in init self._set_colors() File "/usr/lib/python3.8/site-packages/ansible_navigator/ui_framework/curses_window.py", line 189, in _set_colors with open(self._ui_config.terminal_colors_path) as data_file: FileNotFoundError: [Errno 2] No such file or directory: '/root/.local/share/ansible_navigator/themes/terminal_colors.json'

`

patrick-tang-redhat avatar Apr 07 '22 18:04 patrick-tang-redhat

You can use the supported ee by supplying it on the command line:

ansible-navigator --eei registry.redhat.io/ansible-automation-platform-21/ee-supported-rhel8:latest

Or putting it in a ansible-navigator settings file.

The upstream releases will point to the creator-ee. I need to see what is being done during the downstream RPM build, they might be using a settings file or default being changed.

cidrblock avatar Apr 07 '22 18:04 cidrblock

BTW, sorry you are having trouble here... we'll get this sorted out

cidrblock avatar Apr 07 '22 18:04 cidrblock

@cidrblock I just tried with --eei to specify ee-supported image, but it just ran fine in venv. So not sure why i got issue in non-venv . like below:

(venv) [root@rhel8 navigator-test]# ansible-navigator collections --eei registry.redhat.io/ansible-automation-platform-21/ee-supported-rhel8:latest

Execution environment image and pull policy overview

Execution environment image name: registry.redhat.io/ansible-automation-platform-21/ee-supported-rhel8:latest Execution environment image tag: latest Execution environment pull arguments: None Execution environment pull policy: tag Execution environment pull needed: True

Updating the execution environment

Running the command: podman pull registry.redhat.io/ansible-automation-platform-21/ee-supported-rhel8:latest Trying to pull registry.redhat.io/ansible-automation-platform-21/ee-supported-rhel8:latest... Getting image source signatures Checking if image destination supports signatures Copying blob 4aa6faa2acb0 skipped: already exists
Copying blob 4a3604715398 skipped: already exists
Copying blob 6ccfcef17d4b skipped: already exists
Copying blob 916e0bda4eea skipped: already exists
Copying blob 510abfcdf6bc skipped: already exists
Copying config 43cfa4a7f9 done
Writing manifest to image destination Storing signatures 43cfa4a7f9f7176c797dfc05a211cc754e0cb29e1ffd0d2a1ad954a8dd690b81 (venv) [root@rhel8 navigator-test]# python3 --version Python 3.9.6 (venv) [root@rhel8 navigator-test]# ansible-navigator --version ansible-navigator 2.0.0b0

patrick-tang-redhat avatar Apr 07 '22 19:04 patrick-tang-redhat

Something about the way it was installed outside the virtual environment...

If you want to use it without the virtual environment, you can

deactivate the virtual environment $ deactivate

and then remove the RPM installed ansible-navigator and remove the previously installed pip package $ pip uninstall ansible-navigator

and try to install if fresh:

$ pip install ansible-navigator==2.0.0b0 --user

this might work, I haven't tried a pip install on top of an RPM...

cidrblock avatar Apr 07 '22 19:04 cidrblock

@patrick-tang-redhat JSYK:

pip install ansible-navigator==2.0.0rc1

Just released.

Let me know if it's ok to close this issue now :)

cidrblock avatar Apr 07 '22 19:04 cidrblock

@cidrblock sorry. was caught in customer work. I uninstalled the rpm, and used pip to install 2.0.0rc1 ('pip3 install ansible-navigator==2.0.0rc1'), but got error:

` [root@rhel8 ~]# /usr/local/bin/ansible-navigator --version ansible-navigator 2.0.0rc1

[root@rhel8 ~]# /usr/local/bin/ansible-navigator collections --eei ee-supported-rhel8

Execution environment image and pull policy overview

Execution environment image name: ee-supported-rhel8:latest Execution environment image tag: latest Execution environment pull arguments: None Execution environment pull policy: tag Execution environment pull needed: True

Updating the execution environment

Running the command: podman pull ee-supported-rhel8:latest Trying to pull registry.redhat.io/ansible-automation-platform-21/ee-supported-rhel8:latest... Getting image source signatures Checking if image destination supports signatures Copying blob 4a3604715398 skipped: already exists
Copying blob 4aa6faa2acb0 skipped: already exists
Copying blob 6ccfcef17d4b skipped: already exists
Copying blob 916e0bda4eea skipped: already exists
Copying blob 510abfcdf6bc skipped: already exists
Copying config 43cfa4a7f9 done
Writing manifest to image destination Storing signatures 43cfa4a7f9f7176c797dfc05a211cc754e0cb29e1ffd0d2a1ad954a8dd690b81 Traceback (most recent call last): File "/usr/local/bin/ansible-navigator", line 8, in sys.exit(main()) File "/usr/local/lib/python3.9/site-packages/ansible_navigator/cli.py", line 159, in main run_return = run(args) File "/usr/local/lib/python3.9/site-packages/ansible_navigator/cli.py", line 97, in run wrapper(ActionRunner(args=args).run) File "/usr/lib64/python3.9/curses/init.py", line 94, in wrapper return func(stdscr, *args, **kwds) File "/usr/local/lib/python3.9/site-packages/ansible_navigator/action_runner.py", line 90, in run self.initialize_ui(DEFAULT_REFRESH) File "/usr/local/lib/python3.9/site-packages/ansible_navigator/action_runner.py", line 73, in initialize_ui self._ui = UserInterface( File "/usr/local/lib/python3.9/site-packages/ansible_navigator/ui_framework/ui.py", line 154, in init super().init(ui_config=ui_config) File "/usr/local/lib/python3.9/site-packages/ansible_navigator/ui_framework/curses_window.py", line 64, in init self._set_colors() File "/usr/local/lib/python3.9/site-packages/ansible_navigator/ui_framework/curses_window.py", line 198, in _set_colors with open(self._ui_config.terminal_colors_path, encoding="utf-8") as fh: FileNotFoundError: [Errno 2] No such file or directory: '/usr/share/ansible_navigator/themes/terminal_colors.json'

`

I guess I just wait for the new release.

patrick-tang-redhat avatar Apr 08 '22 01:04 patrick-tang-redhat

hmmm I'm wondering if the RPM uninstall didn't remove some of the directories

try deleting the /usr/share/ansible_navigator directory

cidrblock avatar Apr 08 '22 12:04 cidrblock

I uninstall rpm ansible-navigator then install 2.0.0rc1 with pip. So yum should not remove any rc1 directories. When I was uninstalling, yum removed some python38 dependencies together:

` Removing: ansible-navigator noarch 1.1.0-1.el8ap @ansible-automation-platform 1.1 M Removing unused dependencies: python38-importlib-resources noarch 1.1.0-2.el8ap @ansible-automation-platform 109 k python38-onigurumacffi x86_64 1.1.0-1.el8ap @ansible-automation-platform 45 k

Transaction Summary

`

patrick-tang-redhat avatar Apr 08 '22 14:04 patrick-tang-redhat

@cidrblock After removing /usr/share/ansible_navigator directory, I'm able to run new rc1 successfully to see roles and its related docs. What is explanation about the removed directory? and similar '~/.local/share/ansible_navigator' directory when using 'pip install --user'.

patrick-tang-redhat avatar Apr 08 '22 14:04 patrick-tang-redhat

It seems the RPM uninstall doesn't remove the directory it created, so when the pip installed version runs it finds that directory, even though it is empty, and tries to load from it. We are trying to get rid of the share directory entirely, which is the real long term fix, given that I'm not sure there is a reason to do anything right now. I think it would be rare for someone to be switching from RPM to pip, unless they are testing or developing...

cidrblock avatar Apr 18 '22 14:04 cidrblock

It seems the RPM uninstall doesn't remove the directory it created, so when the pip installed version runs it finds that directory, even though it is empty, and tries to load from it. We are trying to get rid of the share directory entirely, which is the real long term fix, given that I'm not sure there is a reason to do anything right now. I think it would be rare for someone to be switching from RPM to pip, unless they are testing or developing...

@cidrblock Thanks for the explanation.

patrick-tang-redhat avatar Apr 18 '22 20:04 patrick-tang-redhat

@cidrblock Since the change is planned and on the way, I think this can be closed.

patrick-tang-redhat avatar May 04 '22 13:05 patrick-tang-redhat

Resolved by #1059

shatakshiiii avatar Nov 04 '22 15:11 shatakshiiii