haystack-core-integrations
haystack-core-integrations copied to clipboard
fix: normalising ChatGenerators output
Related Issues
- fixes #7687
Proposed Changes:
- Something change in AmazonBedrock and the PR I merged had no effect
How did you test it?
unit tests, integration tests, manual verification, instructions for manual tests
A few things I noticed when testing the ChatGenarators
Test coverage
generators/amazon_bedrock/chat/adapters.pyis at 59%generators/amazon_bedrock/chat/chat_generator.pyis at 51%
Some integrations tests are still being skipped, see the CI logs for details
cmd [1] | coverage run -m pytest -m integration --reruns 3 --reruns-delay 30 -x
============================= test session starts ==============================
platform linux -- Python 3.9.19, pytest-8.3.2, pluggy-1.5.0
rootdir: /home/runner/work/haystack-core-integrations/haystack-core-integrations/integrations/amazon_bedrock
configfile: pyproject.toml
plugins: rerunfailures-14.0, anyio-4.4.0
collected 152 items / 140 deselected / 12 selected
tests/test_chat_generator.py::TestMistralAdapter::test_default_inference_params[mistral.mistral-7b-instruct-v0:2] SKIPPED [ 8%]
tests/test_chat_generator.py::TestMistralAdapter::test_default_inference_params[mistral.mixtral-8x7b-instruct-v0:1] SKIPPED [ 16%]
tests/test_chat_generator.py::TestMistralAdapter::test_default_inference_params[mistral.mistral-large-2402-v1:0] SKIPPED [ 25%]
Several warnings about parameters being ignored
WARNING haystack_integrations.components.generators.amazon_bedrock.chat.adapters:adapters.py:76 Parameter 'top_k' is not allowed and will be ignored.
WARNING haystack_integrations.components.generators.amazon_bedrock.chat.adapters:adapters.py:76 Parameter 'stop_sequences' is not allowed and will be ignored.
WARNING haystack_integrations.components.generators.amazon_bedrock.chat.adapters:adapters.py:76 Parameter 'stream' is not allowed and will be ignored.
Checklist
- I have read the contributors guidelines and the code of conduct
- I have updated the related issue with new insights and changes
- I added unit tests and updated the docstrings
- I've used one of the conventional commit types for my PR title:
fix:,feat:,build:,chore:,ci:,docs:,style:,refactor:,perf:,test:.
@vblagoje @anakin87 regarding the issues I raised in the PR comment - I will open an issue for it, you agree?
On test coverage, etc.: see #801
In short, we split unit and integration tests to make integration tests run only if authenticated (not from forks).
- This causes the coverage numbers to be strange and probably incorrect.
- Selected tests are different depending on the unit/integration workflow step.
ok I see the rational do to that, but since I couldn't successfully authenticate locally to test everything, I'm not sure something will fail when the full tests run on main
@anakin87 fixed
This test is wrong https://github.com/deepset-ai/haystack-core-integrations/blob/55c65af5ac338f0678537125d6fd0e92dfa689b6/integrations/amazon_bedrock/tests/test_chat_generator.py#L231
meta should be a dict: https://github.com/deepset-ai/haystack/blob/35b1215b00b88b2654ea50de2453e5f852e72328/haystack/dataclasses/chat_message.py#L33
I think you can fix the test and add back the previous suggestion.