Bug/semaphore lock

[vasilypht]

Related Issue

Checklist

• [x] I have read the CONTRIBUTING guidelines.
• [ ] I have added tests to cover my changes.
• [x] I have updated the documentation (docs folder) accordingly.

Additional Notes

Hi!

When running multiple jobs asynchronously, a semaphore lock occurs. Multiple jobs will increment the semaphore counter to >3, and after the first job completes, the lock will remain.

I suggest using a mutex, so all new jobs will simply wait for it to be released and then proceed quietly.

Example code with locking:

import uuid
import asyncio

from infinity_emb import AsyncEmbeddingEngine, EngineArgs


async def run_job_a(engine: AsyncEmbeddingEngine, texts: list[str]) -> list[list[float]]:
    task_id = uuid.uuid4()
    print(f"{task_id=} started")
    async with engine:
        result, usage = await engine.embed(sentences=texts)
    print(f"{task_id=} completed")
    return result


async def main() -> None:
    args = EngineArgs(
        model_name_or_path="/home/models/e5-small-v2",
    )
    engine = AsyncEmbeddingEngine.from_args(engine_args=args)
    
    outputs = await asyncio.gather(
        run_job_a(engine=engine, texts=["a", "b", "c", "d"]),
        run_job_a(engine=engine, texts=["a", "b", "c", "d"]),
        run_job_a(engine=engine, texts=["a", "b", "c", "d"])
    )

if __name__ == "__main__":
    asyncio.run(main())

I understand that the behavior with constantly turning on/off the engine is incorrect, but suddenly someone wants to do this)

[michaelfeil]

Thanks, looking into it.

[codecov-commenter]

:warning: Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

:x: Patch coverage is 93.33333% with 1 line in your changes missing coverage. Please review. :white_check_mark: Project coverage is 79.49%. Comparing base (fcb951c) to head (07dfe74).

Files with missing lines	Patch %	Lines
libs/infinity_emb/infinity_emb/engine.py	93.33%	1 Missing :warning:
:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #647      +/-   ##
==========================================
- Coverage   79.54%   79.49%   -0.06%     
==========================================
  Files          43       43              
  Lines        3495     3501       +6     
==========================================
+ Hits         2780     2783       +3     
- Misses        715      718       +3     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[michaelfeil]

@vasilypht Can you comment on This PR will cause immediate deadlocks in production and must not be merged?

[vasilypht]

@vasilypht Can you comment on This PR will cause immediate deadlocks in production and must not be merged?

The review was on the first commit, and there really was a not very good decision there that could have caused the blocking.

[michaelfeil]

@codex review