Store Request and Group IDs in backend key value store

[hacktobeer]

We currently use key/value stores as backend. This stores the key as 'TurbiniaTask:[task_id]' with a JSON blob as value containing the details (including request id and group id). If you want to query on request or group ID we loop through all keys and parse the value to get the relevant task IDs.

We could store the RequestID with a value list of TaskIDs and the GroupID with a value list of Request IDs to make status fetching faster.

[jleaniz]

Looking into implemting this change. I will use this issue to cover any related bugs for simplicity.

The to_json() method in the TurbiniaRequest class needs to handle invalid evidence objects. It expects the evidence attribute to be a list, not an Evidence object. The name of the attribute is a bit misleading here as well and should consider renaming it to evidence_list or similar.

>>> r = RawDisk(source_path='/evidence/artifact_disk.dd')
>>> t = TurbiniaRequest(evidence=r, requester='leaniz')
>>> t
<turbinia.message.TurbiniaRequest object at 0x7fe97a2a0310>
>>> t.to_json()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/workspaces/turbinia/turbinia/message.py", line 69, in to_json
    serializable['evidence'] = [x.serialize() for x in serializable['evidence']]
TypeError: 'RawDisk' object is not iterable

Passing a list of Evidence objects seems to work as intended.

>>> t = TurbiniaRequest(evidence=[r], requester='leaniz')
>>> t.to_json()
'{"request_id": "3af50f3f6dd74baf8edd8ccbf2c7e1ec", "group_id": "ca6d18a49df54f9fa7bd5191eef1c763", "requester": "leaniz", "recipe": {"globals": {}}, "context": {}, "evidence": [{"copyable": false, "config": {}, "context_dependent": false, "cloud_only": false, "description": null, "size": null, "mount_path": null, "credentials": [], "source": null, "source_path": "/evidence/artifact_disk.dd", "tags": {}, "request_id": null, "has_child_evidence": false, "parent_evidence": null, "save_metadata": false, "resource_tracked": false, "resource_id": null, "local_path": "/evidence/artifact_disk.dd", "processed_by": [], "type": "RawDisk", "_name": null, "saved_path": null, "saved_path_type": null, "state": {"1": false, "2": false, "3": false, "4": false}, "device_path": null}], "group_name": "", "reason": "", "all_args": "", "type": "TurbiniaRequest"}'