ara icon indicating copy to clipboard operation
ara copied to clipboard

Improve exception handling in ara_record

Open dmsimard opened this issue 3 years ago • 1 comments

What is the issue ?

There's been a few cases of users bumping into scary hard-to-parse tracebacks when using ara_record, like this one:

Failed to get on /api/v1/plays?uuid=5983f1d4-ebb4-c774-a03c-0000000000a0: {'params': {}}
The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/ansible/executor/task_executor.py", line 158, in run
    res = self._execute()
  File "/usr/lib/python3/dist-packages/ansible/executor/task_executor.py", line 589, in _execute
    result = self._handler.run(task_vars=variables)
  File "/home/zen/.local/lib/python3.8/site-packages/ara/plugins/action/ara_record.py", line 199, in run
    playbook_id = play["results"][0]["playbook"]
KeyError: 'results'
fatal: [SERVER123]: FAILED! => 
  msg: Unexpected failure during module execution.
  stdout: ''
[WARNING]: Failure using method (v2_runner_on_failed) in callback plugin (<ansible.plugins.callback.ara_default.CallbackModule object at 0x7fbe063800a0>): HTTPSConnectionPool(host='ara-ansible-prod.mydomain.firm', port=443): Max
retries exceeded with url: /api/v1/results (Caused by SSLError(SSLError("read error: Error([('SSL routines', 'ssl3_get_record', 'decryption failed or bad record mac')])")))
Callback Exception: 
  File "/usr/lib/python3/dist-packages/ansible/executor/task_queue_manager.py", line 450, in send_callback
    method(*new_args, **kwargs)
   File "/home/zen/.local/lib/python3.8/site-packages/ara/plugins/callback/ara_default.py", line 391, in v2_runner_on_failed
    self._submit_thread("task", self._load_result, result, "failed", **kwargs)
   File "/home/zen/.local/lib/python3.8/site-packages/ara/plugins/callback/ara_default.py", line 249, in _submit_thread
    func(*args, **kwargs)
   File "/home/zen/.local/lib/python3.8/site-packages/ara/plugins/callback/ara_default.py", line 584, in _load_result
    self.result = self.client.post(
   File "/home/zen/.local/lib/python3.8/site-packages/ara/clients/http.py", line 115, in post
    return self._request("post", endpoint, **kwargs)
   File "/home/zen/.local/lib/python3.8/site-packages/ara/clients/http.py", line 93, in _request
    response = func(url, **kwargs)
   File "/home/zen/.local/lib/python3.8/site-packages/ara/clients/http.py", line 69, in post
    return self._request("post", url, data=json.dumps(payload))
   File "/home/zen/.local/lib/python3.8/site-packages/ara/clients/http.py", line 57, in _request
    return self.http.request(method, self.endpoint + url, timeout=self.timeout, **payload)
   File "/usr/lib/python3/dist-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
   File "/usr/lib/python3/dist-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
   File "/usr/lib/python3/dist-packages/requests/adapters.py", line 514, in send
    raise SSLError(e, request=request)

Where, really, the error is just Failed to get on /api/v1/plays?uuid=5983f1d4-ebb4-c774-a03c-0000000000a0.

In at least one case where it's come up, it was because the user had set up filtering on HTTP methods so that only POST and PATCH were allowed, thus resulting in the GET to /api/v1/plays to fail.

For context, ara_record needs to query the API with it's play uuid in order to find out which playbook it is running from so it can associate the record to the right playbook.

What should be happening ?

We should catch the exception and print something friendlier so we can point the user in the right direction.

dmsimard avatar Mar 22 '22 15:03 dmsimard

Is there a reproductible case for that ? I can fix this and submit a PR if you want ^^.

Maybe we can find a use case with the demo server ?

DataHearth avatar Apr 02 '22 15:04 DataHearth