ara
ara copied to clipboard
ansible-community
WIP: ui: Add a tasks page
This adds a new task "index" page much like the one for playbooks and hosts. It is modelled after /api/v1/tasks.
It's up for testing purposes on demo.recordsansible.org/tasks right now and looks like this:
Thanks for adding this @dmsimard this is a game changer for sifting through a complex play across a large number of hosts!
I like this as well! But would love to see a playbook name/ path column and a filter for playbook name/ path as well. You will have same tasks in different playbooks, so it will be really helpfull to directly see which playbook it is.
Btw for the hosts index view I also miss that, as already mentioned in #331.
@hille721 yes, I think there is work to do to make the pages have a consistent feel in general.
What bothers me is that the status is moderately misleading because it can only be one of the following: https://github.com/ansible-community/ara/blob/f366c9f9f2a06311dd67449f1124dca0674563c2/ara/api/models.py#L202
There is no concept of a failed task: only whether it's completed or not. The checkmark could be green even if there are failed results in the task. Not great.
Computing the status of a task server side would mean looking up each result of a task and return failed if there is a failure. Expensive and slow to do at scale.
The callback can determine the status of a task because there are specific hooks that are triggered on failure like: https://github.com/ansible-community/ara/blob/f366c9f9f2a06311dd67449f1124dca0674563c2/ara/plugins/callback/ara_default.py#L390 or: https://github.com/ansible-community/ara/blob/f366c9f9f2a06311dd67449f1124dca0674563c2/ara/plugins/callback/ara_default.py#L399
This would mean adding a failed task status to the model and then implementing the logic in the callback to set the status of the task accordingly when updating it's status:
https://github.com/ansible-community/ara/blob/f366c9f9f2a06311dd67449f1124dca0674563c2/ara/plugins/callback/ara_default.py#L415-L429
I thought that this could be tricky when running with the free strategy and it reminded me that we haven't fixed _end_task when running with the free strategy as per the discussions in https://github.com/ansible-community/ara/pull/268.
There is a bit of work to do :)
I forgot to say why I pulled this from 1.5.8: it sort of works but as mentioned in my last comment, there needs to be some changes under the hood to make it a better implementation. The changes are significant enough to be in a larger version bump so we can ship some minor updates and bug fixes out sooner.
I forgot to say why I pulled this from 1.5.8: it sort of works but as mentioned in my last comment, there needs to be some changes under the hood to make it a better implementation.
The changes are significant enough to be in a larger version bump so we can ship some minor updates and bug fixes out sooner.
Thanks for your dedication to delivering quality features and updates @dmsimard
There is no concept of a failed task: only whether it's completed or not. The checkmark could be green even if there are failed results in the task. Not great.
I realized we have the same issue with plays since there are only completed or running statuses. I'll focus on tasks first.
I've rebased the patch on top of 1.5.8rc1 and included a first iteration at the API and callback implementation for working with the new "failed" task status. It works though I will review and add at least some tests, most notably for the free strategy.
It looks like this:
Note an interesting case in the demo -- there is a failed task but the playbook is successful. How come ? When you look at the task file, it's because it is later rescued since the task is inside a block with a rescue and there's no ignore_errors.
The callback will only flip the status to "failed" if ignore_errors is unset or false.
Build succeeded.
- ara-tox-py3 : SUCCESS in 4m 35s
- ara-tox-linters : SUCCESS in 6m 53s
- ara-basic-ansible-core-devel : SUCCESS in 6m 38s (non-voting)
- ara-basic-ansible-5 : SUCCESS in 8m 51s
- ara-basic-ansible-core-2.12 : SUCCESS in 6m 36s
- ara-basic-ansible-core-2.11 : SUCCESS in 5m 34s
- ara-basic-ansible-base-2.10 : SUCCESS in 4m 53s
- ara-basic-ansible-2.9 : SUCCESS in 5m 14s
- ara-container-images : SUCCESS in 10m 46s
![softwarefactory-project-zuul[bot] avatar](https://avatars.githubusercontent.com/in/6924?v=4 "Published by a pub.dev verified publisher"){kind=small}
I saw an opportunity to add a link to the new host and task search pages from the numbers on the playbook index so you can easily search for the hosts and tasks for a particular playbook:
Build succeeded.
- ara-tox-py3 : SUCCESS in 4m 37s
- ara-tox-linters : SUCCESS in 6m 49s
- ara-basic-ansible-core-devel : SUCCESS in 8m 11s (non-voting)
- ara-basic-ansible-5 : SUCCESS in 5m 54s
- ara-basic-ansible-core-2.12 : SUCCESS in 4m 50s
- ara-basic-ansible-core-2.11 : SUCCESS in 4m 58s
- ara-basic-ansible-base-2.10 : SUCCESS in 5m 15s
- ara-basic-ansible-2.9 : SUCCESS in 5m 06s
- ara-container-images : SUCCESS in 11m 54s
It works though I will review and add at least some tests, most notably for the free strategy.
I was testing what this looked like with the free strategy and caught a somewhat expected bug where there is a failure in one of the results of the task but the task result was successful. Will try to see if I can fix it.
--
I've updated the database migration to include a script that will iterate over tasks and update the status to failed if there are any failed or unreachable results for that task.
I haven't yet found a solution to the free strategy issue that didn't involve making one or more API calls which I am trying to avoid in the best interest of performance. I am going to file it under known issues for now until we can spend more time fixing issues in the free strategy recording particularly.
I think this should be good to go as a first iteration. I will make a backup of the demo.recordsansible.org database and update it with the latest revision of this PR.
Build failed.
- ara-tox-py3 : SUCCESS in 7m 01s
- ara-tox-linters : SUCCESS in 5m 42s
- ara-basic-ansible-core-devel : SUCCESS in 8m 13s (non-voting)
- ara-basic-ansible-5 : SUCCESS in 8m 04s
- ara-basic-ansible-core-2.12 : SUCCESS in 7m 09s
- ara-basic-ansible-core-2.11 : SUCCESS in 7m 07s
- ara-basic-ansible-base-2.10 : SUCCESS in 7m 52s
- ara-basic-ansible-2.9 : FAILURE in 9m 31s
- ara-container-images : SUCCESS in 15m 15s
I ran the migration on the live demo with 489 920 tasks and 743 983 results (across 3483 playbooks).
It took almost 11 minutes to run on a fairly fast virtual machine backed by a local mariadb server:
$ time ./virtualenv/bin/ara-manage migrate
[ara] Using settings file: /home/centos/.ara/server/settings.yaml
Operations to perform:
Apply all migrations: admin, api, auth, contenttypes, db, sessions
Running migrations:
Applying api.0011_task_status... status updated to "failed" for 505 tasks due to failed or unreachable results
OK
Applying auth.0012_alter_user_first_name_max_length... OK
real 10m47.404s
user 8m4.064s
sys 0m42.223s
I wonder if it would be faster to check for failed or unreachable results and then ensure that their parent task's status is set to failed.
Oh wow, I don't know what I was thinking... it's much faster to only iterate through failed or unreachable results instead of iterating over every task:
$ time ./virtualenv/bin/ara-manage migrate
[ara] Using settings file: /home/centos/.ara/server/settings.yaml
Operations to perform:
Apply all migrations: admin, api, auth, contenttypes, db, sessions
Running migrations:
Applying api.0011_task_status... status updated to "failed" for 505 tasks due to failed or unreachable results
OK
Applying auth.0012_alter_user_first_name_max_length... OK
real 0m3.069s
user 0m1.862s
sys 0m0.201s
3 seconds instead of 11 minutes :exploding_head:
Enough iterating for today but I do want to point out before I forget: running the migrations updates the "updated" field for the tasks that are updated which... makes sense but can perhaps provide confusing results when looking at the tasks of a playbook:
Build failed.
- ara-tox-py3 : SUCCESS in 10m 23s
- ara-tox-linters : FAILURE in 7m 59s
- ara-basic-ansible-core-devel : SUCCESS in 7m 23s (non-voting)
- ara-basic-ansible-5 : SUCCESS in 7m 58s
- ara-basic-ansible-core-2.12 : SUCCESS in 6m 03s
- ara-basic-ansible-core-2.11 : FAILURE in 7m 58s
- ara-basic-ansible-base-2.10 : SUCCESS in 5m 48s
- ara-basic-ansible-2.9 : SUCCESS in 7m 23s
- ara-container-images : SUCCESS in 14m 49s
While using the tasks page I realized I forgot to add a couple fields:
We should also be able to search by task file path and lineno, handler (boolean), as well as file (to view all the tasks from a file?) and play: https://github.com/ansible-community/ara/blob/db8243c3af938ece12c9cd59dd7fe4d9a711b76d/ara/api/models.py#L202-L213
We can't search by tags yet unfortunately. With the power of hindsight, the implementation shouldn't have been using compression (which was meant for larger fields like result content) and which makes it harder (and much more expensive) to search text.
I'm hoping to improve that eventually by relying on django's 3.2's new json fields but it will have to be in a "very major" release since it will require significant database and API changes.
Just rebased the PR because the branch started to be old/stale.
That issue about the last updated field being bumped during the migration kind of bothers me but at the same time, it's doing what is intended.
I guess we should change the field in the table to be the date of creation instead of the date of update ?
Build failed.
:x: ara-tox-py3 FAILURE in 3m 43s :x: ara-tox-linters FAILURE in 3m 50s :x: ara-basic-ansible-core-devel FAILURE in 2m 36s (non-voting) :x: ara-basic-ansible-6 FAILURE in 3m 08s :x: ara-basic-ansible-core-2.13 FAILURE in 3m 03s :x: ara-basic-ansible-core-2.12 FAILURE in 2m 59s :x: ara-basic-ansible-core-2.11 FAILURE in 3m 01s :x: ara-basic-ansible-2.9 FAILURE in 4m 33s :x: ara-container-images FAILURE in 5m 25s
CommandError: Conflicting migrations detected; multiple leaf nodes in the migration graph: (0011_play_label_name_max_length, 0011_task_status in api).
To fix them run 'python manage.py makemigrations --merge'
Build failed.
:heavy_check_mark: ara-tox-py3 SUCCESS in 8m 36s :x: ara-tox-linters FAILURE in 8m 10s :heavy_check_mark: ara-basic-ansible-core-devel SUCCESS in 5m 48s (non-voting) :heavy_check_mark: ara-basic-ansible-6 SUCCESS in 5m 26s :x: ara-basic-ansible-core-2.13 FAILURE in 4m 49s :heavy_check_mark: ara-basic-ansible-core-2.12 SUCCESS in 4m 51s :heavy_check_mark: ara-basic-ansible-core-2.11 SUCCESS in 5m 47s :x: ara-basic-ansible-2.9 FAILURE in 8m 46s :heavy_check_mark: ara-container-images SUCCESS in 11m 22s
I've iterated on a batch of various changes, it probably needs some cleanup before landing but it's largely good to go for this first release.
This is what it looks like now (open gif in new tab if it's too small):
Build failed.
:heavy_check_mark: ara-tox-py3 SUCCESS in 3m 35s :x: ara-tox-linters FAILURE in 3m 13s :heavy_check_mark: ara-basic-ansible-core-devel SUCCESS in 4m 55s (non-voting) :x: ara-basic-ansible-6 FAILURE in 3m 52s :heavy_check_mark: ara-basic-ansible-core-2.13 SUCCESS in 4m 34s :heavy_check_mark: ara-basic-ansible-core-2.12 SUCCESS in 5m 53s :heavy_check_mark: ara-basic-ansible-core-2.11 SUCCESS in 4m 48s :heavy_check_mark: ara-basic-ansible-2.9 SUCCESS in 5m 52s :heavy_check_mark: ara-container-images SUCCESS in 10m 24s
Build succeeded.
:heavy_check_mark: ara-tox-py3 SUCCESS in 3m 51s :heavy_check_mark: ara-tox-linters SUCCESS in 4m 11s :heavy_check_mark: ara-basic-ansible-core-devel SUCCESS in 5m 30s (non-voting) :heavy_check_mark: ara-basic-ansible-6 SUCCESS in 5m 16s :heavy_check_mark: ara-basic-ansible-core-2.13 SUCCESS in 5m 18s :heavy_check_mark: ara-basic-ansible-core-2.12 SUCCESS in 6m 35s :heavy_check_mark: ara-basic-ansible-core-2.11 SUCCESS in 5m 09s :heavy_check_mark: ara-basic-ansible-2.9 SUCCESS in 5m 20s :heavy_check_mark: ara-container-images SUCCESS in 12m 10s
I've deployed this PR to https://demo.recordsansible.org/tasks, can you let me know what you think ?
Build succeeded.
:heavy_check_mark: ara-tox-py3 SUCCESS in 10m 15s :heavy_check_mark: ara-tox-linters SUCCESS in 11m 26s :heavy_check_mark: ara-basic-ansible-core-devel SUCCESS in 13m 27s (non-voting) :heavy_check_mark: ara-basic-ansible-6 SUCCESS in 13m 15s :heavy_check_mark: ara-basic-ansible-core-2.13 SUCCESS in 12m 13s :heavy_check_mark: ara-basic-ansible-core-2.12 SUCCESS in 12m 41s :heavy_check_mark: ara-basic-ansible-core-2.11 SUCCESS in 13m 46s :heavy_check_mark: ara-basic-ansible-2.9 SUCCESS in 12m 57s :heavy_check_mark: ara-container-images SUCCESS in 18m 22s
Build succeeded.
:heavy_check_mark: ara-tox-py3 SUCCESS in 4m 31s :heavy_check_mark: ara-tox-linters SUCCESS in 3m 23s :heavy_check_mark: ara-basic-ansible-core-devel SUCCESS in 5m 22s (non-voting) :heavy_check_mark: ara-basic-ansible-6 SUCCESS in 5m 29s :heavy_check_mark: ara-basic-ansible-core-2.13 SUCCESS in 6m 48s :heavy_check_mark: ara-basic-ansible-core-2.12 SUCCESS in 6m 01s :heavy_check_mark: ara-basic-ansible-core-2.11 SUCCESS in 7m 00s :heavy_check_mark: ara-basic-ansible-2.9 SUCCESS in 5m 20s :heavy_check_mark: ara-container-images SUCCESS in 12m 45s
I've cleaned up the history, this is no longer WIP -- I've tested it and it is good to land IMO but I'll let it simmer a bit on the demo to see if folks have feedback.
Build succeeded.
:heavy_check_mark: ara-tox-py3 SUCCESS in 4m 25s :heavy_check_mark: ara-tox-linters SUCCESS in 3m 43s :heavy_check_mark: ara-basic-ansible-core-devel SUCCESS in 6m 05s (non-voting) :heavy_check_mark: ara-basic-ansible-6 SUCCESS in 6m 20s :heavy_check_mark: ara-basic-ansible-core-2.13 SUCCESS in 7m 16s :heavy_check_mark: ara-basic-ansible-core-2.12 SUCCESS in 6m 46s :heavy_check_mark: ara-basic-ansible-core-2.11 SUCCESS in 5m 07s :heavy_check_mark: ara-basic-ansible-2.9 SUCCESS in 6m 25s :heavy_check_mark: ara-container-images SUCCESS in 13m 51s