deepset-ai
added partial test setup #1659
Related Issues
- Fixes #1659
Proposed Changes
How Did You Test It?
All unit tests passed after running hatch run test with a brand new MongoDB Atlas cluster.
Notes for the Reviewer
I encountered the following error:
pymongo.errors.OperationFailure: The maximum number of FTS indexes has been reached for this instance size., full error: {'ok': 0.0, 'errmsg': 'The maximum number of FTS indexes has been reached for this instance size.', 'code': 20, 'codeName': 'IllegalOperation', '$clusterTime': {'clusterTime': Timestamp(1745268638, 17), 'signature': {'hash':
It seems I can't create more than 3 full-text search (FTS) indexes on the free-tier MongoDB Atlas cluster, but more are needed.
Does anyone have suggestions on how to work around this limitation?
@MetroCat69 Thanks for opening this PR. We are not aware of any good way to work around the limit of 3 full-text search (FTS) indexes on the free-tier MongoDB Atlas cluster. So we need to make sure we only use 3.
There are some smaller linter issues that need to be fixed: https://github.com/deepset-ai/haystack-core-integrations/actions/runs/14581768683/job/40899798526?pr=1663
You can run the linter locally with hatch run lint:all.
Regarding the renaming of the secret MONGO_CONNECTION_STRING_2, we would need to make sure that we properly pass the secret in this workflow otherwise tests won't use the correct key in the CI.
https://github.com/deepset-ai/haystack-core-integrations/blob/ba0a310269262517bce2b702e4f5bd679b391863/.github/workflows/mongodb_atlas.yml#L26
There are some smaller linter issues that need to be fixed: https://github.com/deepset-ai/haystack-core-integrations/actions/runs/14581768683/job/40899798526?pr=1663
You can run the linter locally withhatch run lint:all.Regarding the renaming of the secret
MONGO_CONNECTION_STRING_2, we need to make sure it's properly passed in the workflow—otherwise, the tests won't use the correct key in CI.https://github.com/deepset-ai/haystack-core-integrations/blob/ba0a310269262517bce2b702e4f5bd679b391863/.github/workflows/mongodb_atlas.yml#L26
Sure, I'm a bit busy over the next few days, so it might take me some time to get to this. Also, I'm not sure it's even possible to use fewer than three indexes, considering we want to test full-text search, cosine similarity, Euclidean similarity, and dot product search.
Maybe we should consider creating and destroying the indexes within the individual tests themselves?
With the free-tier MongoDB Atlas cluster that we currently use in the continuous integration tests, we can't merge this branch right away. I'll close it for now but feel free to reopen it if you find a solution with only three full-text search indexes.