feat(agents): add timeout parameter to chat method
Adds a timeout parameter to the agents.chat() method to handle long-running agent executions. The method now uses _do_request() with a default timeout of 60 seconds, allowing endpoint-specific timeout configuration instead of relying solely on global client timeout settings.
Cursor Agent can help with this pull request. Just @cursor in comments and I'll start working on changes in this branch.
Learn more about Cursor Agents
Codecov Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 90.95%. Comparing base (787926e) to head (05aec6e).
:warning: Report is 1 commits behind head on master.
Additional details and impacted files
@@ Coverage Diff @@
## master #2316 +/- ##
==========================================
- Coverage 90.96% 90.95% -0.02%
==========================================
Files 192 192
Lines 26133 26133
==========================================
- Hits 23773 23768 -5
- Misses 2360 2365 +5
| Files with missing lines | Coverage Δ | |
|---|---|---|
| cognite/client/_api/agents/agents.py | 100.00% <100.00%> (ø) |
:rocket: New features to boost your workflow:
- :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
- :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
I’m okay with introducing a higher timeout setting for these agent APIs that we expect to be «long running»
I’m okay with introducing a higher timeout setting for these agent APIs that we expect to be «long running»
That would be great @haakonvt . How would we do that?
/gemini review
I don't mind letting the user increase the timeout for this API if this is how we want the API to behave, but I would still like to express some concerns around the API design. Having long-running HTTP requests introduces some problems:
- ties up connection resources in the service for a long time
- ties up concurency/rate-limiting resources in the service for a long time
- makes horizontal scaling harder, since you need to wait for long-running ongoing requests to finish
- services on the request path will usually set lower timeouts (e.g. emissary and nginx have 60 or 90 sec by default), so you need to ensure your whole stack allows increased timeouts
Have you considered building an async API for this use case? i.e. one endpoint to start the task and another to poll for results.
How long would the user usually want to bump this? Remembering that I believe there is a general 90 second timeout in the API-gateway already?
@miladh @andeplane
See comments from Erlend and Thorkild.