client-python
client-python copied to clipboard
Handle Connection hangups properly
Description
Any hiccup to the graphql API will cause an API error, and makes calling .list() at scale impractical as it cant hold page / state.
This error handling is in-scope of the SDK
Environment
- OS (where OpenCTI server runs): CentOS Stream 9
- OpenCTI version: 10.5.2
- Other environment details: Scaled Cluster Deployment
Reproducible Steps
Steps to create the smallest reproducible scenario:
- Use the SDK.
Expected Output
Automatically handle retries and connection hangups rather then raise an exception if a retry works
Actual Output
{"timestamp": "2023-09-26T17:04:31.821244Z", "level": "INFO", "name": "pycti.entities", "message": "Listing StixCyberObservables after <REMOVED>"}
5
{"timestamp": "2023-09-26T17:04:32.182713Z", "level": "INFO", "name": "pycti.entities", "message": "Listing StixCyberObservables after <REMOVED>"}
4
{"timestamp": "2023-09-26T17:04:32.325378Z", "level": "INFO", "name": "pycti.api", "message": "upstream connect error or disconnect/reset before headers. reset reason: connection failure, transport failure reason: delayed connect error: 111"}
3
{"timestamp": "2023-09-26T17:04:32.398060Z", "level": "ERROR", "name": "root", "message": "Error in message processing, reporting error to API", "exc_info": "Traceback (most recent call last):\n File \"/usr/local/lib/python3.11/site-packages/pycti/connector/opencti_connector_helper.py\", line 235, in _process_message\n json_data[\"internal\"][\"work_id\"], message_task.result()\n ^^^^^^^^^^^^^^^^^^^^^\n File \"/usr/local/lib/python3.11/concurrent/futures/thread.py\", line 58, in run\n result = self.fn(*self.args, **self.kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/opt/opencti-connector-export-file-csv/export-file-csv.py\", line 333, in _process_message\n entities_list = do_list(\n ^^^^^^^^\n File \"/usr/local/lib/python3.11/site-packages/pycti/entities/opencti_stix_cyber_observable.py\", line 401, in list\n result = self.opencti.query(\n ^^^^^^^^^^^^^^^^^^^\n File \"/usr/local/lib/python3.11/site-packages/pycti/api/opencti_api_client.py\", line 365, in query\n raise ValueError(r.text)\nValueError: upstream connect error or disconnect/reset before headers. reset reason: connection failure, transport failure reason: delayed connect error: 111"}
2
{"timestamp": "2023-09-26T17:04:32.398405Z", "level": "INFO", "name": "pycti.api", "message": "Reporting work update_processed work_622e884a-b3fe-401d-8426-6b27e12f283e_2023-09-26T17:03:46.665Z"}
Additional information
Present since release