agent-lightning
agent-lightning copied to clipboard
fix stream response for anthropic api
fix issue https://github.com/microsoft/agent-lightning/issues/262 for anthropic api via agentlightning.LLMProxy
unit test result:
(agentlightning) [email protected]@GCRAZGDL1513:~/code/local_agl$ pytest -v tests/llm_proxy/test_llm_proxy_gpu.py
================================================================================= test session starts =================================================================================
platform linux -- Python 3.12.3, pytest-8.4.2, pluggy-1.6.0 -- /home/v-gengzhang/code/local_agl/.venv/bin/python
cachedir: .pytest_cache
rootdir: /home/v-gengzhang/code/local_agl
configfile: pyproject.toml
plugins: anyio-4.11.0, hydra-core-1.3.2, langsmith-0.3.45, rerunfailures-16.1, asyncio-1.2.0, typeguard-4.4.4
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 8 items
tests/llm_proxy/test_llm_proxy_gpu.py::test_qwen25_model_sanity PASSED [ 12%]
tests/llm_proxy/test_llm_proxy_gpu.py::test_basic_integration PASSED [ 25%]
tests/llm_proxy/test_llm_proxy_gpu.py::test_multiple_requests_one_attempt PASSED [ 37%]
tests/llm_proxy/test_llm_proxy_gpu.py::test_ten_concurrent_requests PASSED [ 50%]
tests/llm_proxy/test_llm_proxy_gpu.py::test_anthropic_client_compat PASSED [ 62%]
tests/llm_proxy/test_llm_proxy_gpu.py::test_tool_call_roundtrip PASSED [ 75%]
tests/llm_proxy/test_llm_proxy_gpu.py::test_streaming_chunks SKIPPED (Streaming is not supported yet) [ 87%]
tests/llm_proxy/test_llm_proxy_gpu.py::test_anthropic_token_ids PASSED [100%]