openllmetry
openllmetry copied to clipboard
fix(ollama): pre-imported funcs instrumentation failure
- [ ] I have added tests that cover my changes.
- [ ] If adding a new instrumentation or changing an existing one, I've added screenshots from some observability platform showing the change.
- [x] PR name follows conventional commits format:
feat(instrumentation): ...orfix(instrumentation): .... - [ ] (If applicable) I have updated the documentation accordingly.
Solved https://github.com/traceloop/openllmetry/issues/2866
Background In many scenarios related to ollama, we write code like this which is invalid for instrumentation.
from traceloop.sdk import Traceloop
from ollama import chat
Traceloop.init()
Looking at packages/sample-app/.venv/lib/python3.12/site-packages/ollama/__init__.py, the module exports a chat name which is simply an alias for _client.chat, established via chat = _client.chat. Since this assignment happens during the module's initialization, code using from ollama import chat effectively captures the current reference of _client.chat and binds it to the chat name in its own scope.
_client = Client()
generate = _client.generate
chat = _client.chat
embed = _client.embed
Later modifications(applying wrappers for instrumentation) to the source method (Client.chat) do not propagate to these pre-existing imported references. The imported chat continues to point to the object it referenced at the time of import.
Verification
If we import chat(generate) only when use it rather than import it immediately at the entry of the file, the instrumentation works fine.
from traceloop.sdk import Traceloop
import ollama # do not import chat()
Traceloop.init()
...
# instrumentation works
ollama.chat()
...
While our unit tests(packages/opentelemetry-instrumentation-ollama/tests/test_chat.py) follow this pattern, I don't think they're sufficient to catch the pre-imported function instrumentation failure.
Solution
for module_name, module in sys.modules.items():
for attr_name, attr_value in module.__dict__.items():
if attr_name == "generate" and attr_value.__module__.startswith("ollama"):
module.__dict__[attr_name] = InstrumentedFunction("generate")
Using the above code we scan every loaded module and find any attribute named chat (or generate, embeddings, etc.) whose implementation lives in the ollama package, and replace it with our dynamic proxy. That proxy then delegates to the latest ollama.generate under the hood—so no matter how the user imported the function, their calls still get instrumented.
[!IMPORTANT] Fixes instrumentation failure for pre-imported Ollama functions by introducing a proxy class and replacing references in other modules.
- Behavior:
- Fixes instrumentation failure for pre-imported functions in Ollama by replacing references in other modules.
- Introduces
InstrumentedFunctionproxy class to dynamically call latest Ollama function implementations.- Instrumentation:
- Modifies
_instrument()in__init__.pyto iterate throughsys.modulesand replace references to imported Ollama functions withInstrumentedFunction.- Logs replacement actions and handles exceptions with warnings.
This description was created by
for 883dd00502932961bf32ccd0b5bf5a3501899546. You can customize this summary. It will automatically update as commits are pushed.