openllmetry icon indicating copy to clipboard operation
openllmetry copied to clipboard

fix(ollama): pre-imported funcs instrumentation failure

Open minimAluminiumalism opened this issue 5 months ago • 0 comments

  • [ ] I have added tests that cover my changes.
  • [ ] If adding a new instrumentation or changing an existing one, I've added screenshots from some observability platform showing the change.
  • [x] PR name follows conventional commits format: feat(instrumentation): ... or fix(instrumentation): ....
  • [ ] (If applicable) I have updated the documentation accordingly.

Solved https://github.com/traceloop/openllmetry/issues/2866


Background In many scenarios related to ollama, we write code like this which is invalid for instrumentation.

from traceloop.sdk import Traceloop
from ollama import chat

Traceloop.init()

Looking at packages/sample-app/.venv/lib/python3.12/site-packages/ollama/__init__.py, the module exports a chat name which is simply an alias for _client.chat, established via chat = _client.chat. Since this assignment happens during the module's initialization, code using from ollama import chat effectively captures the current reference of _client.chat and binds it to the chat name in its own scope.

_client = Client()

generate = _client.generate
chat = _client.chat
embed = _client.embed

Later modifications(applying wrappers for instrumentation) to the source method (Client.chat) do not propagate to these pre-existing imported references. The imported chat continues to point to the object it referenced at the time of import.

Verification If we import chat(generate) only when use it rather than import it immediately at the entry of the file, the instrumentation works fine.

from traceloop.sdk import Traceloop
import ollama # do not import chat()

Traceloop.init()
...
# instrumentation works
ollama.chat() 
...

While our unit tests(packages/opentelemetry-instrumentation-ollama/tests/test_chat.py) follow this pattern, I don't think they're sufficient to catch the pre-imported function instrumentation failure.

Solution

for module_name, module in sys.modules.items():
    for attr_name, attr_value in module.__dict__.items():
        if attr_name == "generate" and attr_value.__module__.startswith("ollama"):
            module.__dict__[attr_name] = InstrumentedFunction("generate")

Using the above code we scan every loaded module and find any attribute named chat (or generate, embeddings, etc.) whose implementation lives in the ollama package, and replace it with our dynamic proxy. That proxy then delegates to the latest ollama.generate under the hood—so no matter how the user imported the function, their calls still get instrumented.


[!IMPORTANT] Fixes instrumentation failure for pre-imported Ollama functions by introducing a proxy class and replacing references in other modules.

  • Behavior:
    • Fixes instrumentation failure for pre-imported functions in Ollama by replacing references in other modules.
    • Introduces InstrumentedFunction proxy class to dynamically call latest Ollama function implementations.
  • Instrumentation:
    • Modifies _instrument() in __init__.py to iterate through sys.modules and replace references to imported Ollama functions with InstrumentedFunction.
    • Logs replacement actions and handles exceptions with warnings.

This description was created by Ellipsis for 883dd00502932961bf32ccd0b5bf5a3501899546. You can customize this summary. It will automatically update as commits are pushed.

minimAluminiumalism avatar Apr 28 '25 18:04 minimAluminiumalism