Python event timing is weird

[siyangqiu]

🐛 Bug Report

🔎 Describe the Bug The time gap between events is inconsistent. I think we may have a blocking issue with how we post data.

🔄 Reproduction Steps Run this script:

import time


from agentops import AgentOps, Event

ao = AgentOps(api_key="1447cc5f-61b2-4967-ae1c-d30610bc4ca2")
time.sleep(1)
print(time.time())
ao.record(Event("SQIU_event", result="success"))
time.sleep(1)
print(time.time())
ao.record(Event("SQIU_event2", result="success"))
print(time.time())
ao.record(Event("SQIU_event2", result="success"))
print(time.time())
ao.record(Event("SQIU_event2", result="success"))
print(time.time())
ao.record(Event("SQIU_event2", result="success"))
print(time.time())
ao.record(Event("SQIU_event2", result="success"))
print(time.time())
ao.record(Event("SQIU_event2", result="success"))
time.sleep(1.5)
print(time.time())
ao.record(Event("SQIU_event", result="success"))
time.sleep(0.5)
print(time.time())
ao.record(Event("SQIU_event", result="success"))
time.sleep(2)
print(time.time())
ao.record(Event("SQIU_event", result="success"))
ao.end_session(end_state="fail")

The time gap between the events don't line up with the sleep times in the script:

> python add_events.py
1692249866.7725768
1692249867.77273
b'{"events": [{"session_id": "e141b465-378b-4b79-b47c-a4386aaacdc7", "event_type": "SQIU_event", "params": null, "output": null, "result": "success", "tags": null, "timestamp": "2023-08-16T22:24:26.772Z"}]}'
1692249870.3852417
1692249870.3853083
1692249870.385325
1692249870.3853374
1692249870.3853488
1692249871.8857806
b'{"events": [{"session_id": "e141b465-378b-4b79-b47c-a4386aaacdc7", "event_type": "SQIU_event2", "params": null, "output": null, "result": "success", "tags": null, "timestamp": "2023-08-16T22:24:27.772Z"}, {"session_id": "e141b465-378b-4b79-b47c-a4386aaacdc7", "event_type": "SQIU_event2", "params": null, "output": null, "result": "success", "tags": null, "timestamp": "2023-08-16T22:24:30.385Z"}, {"session_id": "e141b465-378b-4b79-b47c-a4386aaacdc7", "event_type": "SQIU_event2", "params": null, "output": null, "result": "success", "tags": null, "timestamp": "2023-08-16T22:24:30.385Z"}, {"session_id": "e141b465-378b-4b79-b47c-a4386aaacdc7", "event_type": "SQIU_event2", "params": null, "output": null, "result": "success", "tags": null, "timestamp": "2023-08-16T22:24:30.385Z"}, {"session_id": "e141b465-378b-4b79-b47c-a4386aaacdc7", "event_type": "SQIU_event2", "params": null, "output": null, "result": "success", "tags": null, "timestamp": "2023-08-16T22:24:30.385Z"}, {"session_id": "e141b465-378b-4b79-b47c-a4386aaacdc7", "event_type": "SQIU_event2", "params": null, "output": null, "result": "success", "tags": null, "timestamp": "2023-08-16T22:24:30.385Z"}]}'
1692249872.8345242
b'{"events": [{"session_id": "e141b465-378b-4b79-b47c-a4386aaacdc7", "event_type": "SQIU_event", "params": null, "output": null, "result": "success", "tags": null, "timestamp": "2023-08-16T22:24:31.885Z"}, {"session_id": "e141b465-378b-4b79-b47c-a4386aaacdc7", "event_type": "SQIU_event", "params": null, "output": null, "result": "success", "tags": null, "timestamp": "2023-08-16T22:24:32.834Z"}]}'
1692249874.835022
b'{"events": [{"session_id": "e141b465-378b-4b79-b47c-a4386aaacdc7", "event_type": "SQIU_event", "params": null, "output": null, "result": "success", "tags": null, "timestamp": "2023-08-16T22:24:34.835Z"}]}'
b'{"session": {"session_id": "e141b465-378b-4b79-b47c-a4386aaacdc7", "init_timestamp": "2023-08-16T22:24:25.772Z", "tags": null, "end_state": "fail", "rating": null, "end_timestamp": "2023-08-16T22:24:34.835Z"}}'

[areibman]

Looking at similar event queue-based event trackers patterns (Kafka, Amplitude, Segment, etc.), it looks like there will always be a delay unless the consumer changes their entire event loop to async. Here's some proposed fixes:

1. Change the post delay from every 1 second to every 10, 30, 60, etc. seconds. This removes some of the "live" capabilities from the dashboard, but it still shows live sessions
2. Add an option to only post sessions + data after runs have been completed.
3. Only post sessions after the queue size is full

No perfect solution in here for Python. Might be better for JS-based agents