agent-lightning icon indicating copy to clipboard operation
agent-lightning copied to clipboard

fix stream response for anthropic api

Open zxgx opened this issue 2 months ago • 0 comments

fix issue https://github.com/microsoft/agent-lightning/issues/262 for anthropic api via agentlightning.LLMProxy

unit test result:

(agentlightning) [email protected]@GCRAZGDL1513:~/code/local_agl$ pytest -v tests/llm_proxy/test_llm_proxy_gpu.py
================================================================================= test session starts =================================================================================
platform linux -- Python 3.12.3, pytest-8.4.2, pluggy-1.6.0 -- /home/v-gengzhang/code/local_agl/.venv/bin/python
cachedir: .pytest_cache
rootdir: /home/v-gengzhang/code/local_agl
configfile: pyproject.toml
plugins: anyio-4.11.0, hydra-core-1.3.2, langsmith-0.3.45, rerunfailures-16.1, asyncio-1.2.0, typeguard-4.4.4
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 8 items                                                                                                                                                                     

tests/llm_proxy/test_llm_proxy_gpu.py::test_qwen25_model_sanity PASSED                                                                                                          [ 12%]
tests/llm_proxy/test_llm_proxy_gpu.py::test_basic_integration PASSED                                                                                                            [ 25%]
tests/llm_proxy/test_llm_proxy_gpu.py::test_multiple_requests_one_attempt PASSED                                                                                                [ 37%]
tests/llm_proxy/test_llm_proxy_gpu.py::test_ten_concurrent_requests PASSED                                                                                                      [ 50%]
tests/llm_proxy/test_llm_proxy_gpu.py::test_anthropic_client_compat PASSED                                                                                                      [ 62%]
tests/llm_proxy/test_llm_proxy_gpu.py::test_tool_call_roundtrip PASSED                                                                                                          [ 75%]
tests/llm_proxy/test_llm_proxy_gpu.py::test_streaming_chunks SKIPPED (Streaming is not supported yet)                                                                           [ 87%]
tests/llm_proxy/test_llm_proxy_gpu.py::test_anthropic_token_ids PASSED                                                                                                          [100%]

zxgx avatar Nov 11 '25 08:11 zxgx