openllmetry icon indicating copy to clipboard operation
openllmetry copied to clipboard

πŸš€ Feature: Install less packages

Open InAnYan opened this issue 8 months ago β€’ 15 comments

Which component is this feature for?

Traceloop SDK

πŸ”– Feature description

I've installed traceloop-sdk with uv, and it installed a lot of packages:

 + opentelemetry-instrumentation-alephalpha==0.39.0
 + opentelemetry-instrumentation-anthropic==0.39.0
 + opentelemetry-instrumentation-bedrock==0.39.0
 + opentelemetry-instrumentation-chromadb==0.39.0
 + opentelemetry-instrumentation-cohere==0.39.0
 + opentelemetry-instrumentation-crewai==0.39.0
 + opentelemetry-instrumentation-google-generativeai==0.39.0
 + opentelemetry-instrumentation-groq==0.39.0
 + opentelemetry-instrumentation-haystack==0.39.0
 + opentelemetry-instrumentation-lancedb==0.39.0
 + opentelemetry-instrumentation-langchain==0.39.0
 + opentelemetry-instrumentation-llamaindex==0.39.0
 + opentelemetry-instrumentation-logging==0.52b1
 + opentelemetry-instrumentation-marqo==0.39.0
 + opentelemetry-instrumentation-milvus==0.39.0
 + opentelemetry-instrumentation-mistralai==0.39.0
 + opentelemetry-instrumentation-ollama==0.39.0
 + opentelemetry-instrumentation-openai==0.39.0
 + opentelemetry-instrumentation-pinecone==0.39.0
 + opentelemetry-instrumentation-qdrant==0.39.0
 + opentelemetry-instrumentation-replicate==0.39.0
 + opentelemetry-instrumentation-requests==0.52b1
 + opentelemetry-instrumentation-sagemaker==0.39.0
 + opentelemetry-instrumentation-sqlalchemy==0.52b1
 + opentelemetry-instrumentation-threading==0.52b1
 + opentelemetry-instrumentation-together==0.39.0
 + opentelemetry-instrumentation-transformers==0.39.0
 + opentelemetry-instrumentation-urllib3==0.52b1
 + opentelemetry-instrumentation-vertexai==0.39.0
 + opentelemetry-instrumentation-watsonx==0.39.0
 + opentelemetry-instrumentation-weaviate==0.39.0
 + opentelemetry-semantic-conventions-ai==0.4.3
 + opentelemetry-util-http==0.52b1

Is there a way to limit number of packages and install only necessary?

🎀 Why is this feature needed ?

I don't really use all of these technologies in my app. This is too much.

Everywhere where openllmetry is mentioned (even your docs), they tell to use this command:

pip install traceloop-sdk

Many projects use this feature: https://stackoverflow.com/a/46775606/10037342, where extensions are listed in square brackets.

✌️ How do you aim to achieve this?

Use optional dependencies, so that only needed packages are installed.

And replace original command with:

pip install traceloop-sdk[full]

Something like that. Free to discuss, free to close πŸ˜„

πŸ”„οΈ Additional Information

No response

πŸ‘€ Have you spent some time to check if this feature request has been raised before?

  • [x] I checked and didn't find similar issue

Are you willing to submit PR?

None

InAnYan avatar Apr 01 '25 09:04 InAnYan

Side note: I see that you can actually install separate packages: https://qdrant.tech/documentation/observability/openllmetry/

InAnYan avatar Apr 01 '25 09:04 InAnYan

Hey @InAnYan, you're right. When we began this OSS we just had 5 packages and it has overgrown a bit. We'll prioritize this but feel free to take a stab at it if you have the capacity - we'd love the contribution :)

nirga avatar Apr 03 '25 15:04 nirga

/assign

brambhattabhishek avatar Apr 04 '25 20:04 brambhattabhishek

You're absolutely right β€” installing traceloop-sdk currently pulls in all optional OpenTelemetry instrumentation packages, even when users only need a subset. This can result in unnecessary bloat, especially for minimal or targeted setups.

It would definitely be more efficient to split these dependencies using extras_require in setup.py or pyproject.toml. For example:

pip install traceloop-sdk[openai] pip install traceloop-sdk[langchain,llamaindex] pip install traceloop-sdk[full] # for everything

This way, users can install only the integrations they need.

If the maintainers are open to this, I’d be happy to help contribute a PR to implement optional dependencies and update the docs accordingly.

Rohan5kumar avatar Apr 07 '25 09:04 Rohan5kumar

you can do it manually or you can use some technics for that and sprate them

gauravbh024 avatar May 23 '25 14:05 gauravbh024

picking this up

amitalokbera avatar May 29 '25 12:05 amitalokbera

@nirga

I'm planning to modularize the dependencies like this, let me know if this looks good to you

  • llm: openai, anthropic, cohere, groq, etc.
  • frameworks: langchain, llamaindex, crewai, haystack, transformers
  • vectorstores: pinecone, qdrant, weaviate, chromadb, lancedb, milvus
  • cloud: bedrock, vertexai, sagemaker, watsonx, google-generativeai
  • minimal: just the base SDK (opentelemetry + pydantic, logging, etc.)
  • all: includes everything above

To handle cases where a user installs the minimal version but tries to use an integration from a missing group, I'm also thinking of adding a small utility like:

from .utils import require_dependency

# Ensure the 'openai' package is installed
require_dependency("openai", "llm")

This would raise a clear error like:

Missing optional dependency 'openai'. Please install with 'traceloop-sdk[llm]' to use this feature.

One drawback is that this check would need to be added in multiple places across the codebase, which isn’t ideal. Open to better suggestions or ideas for a cleaner solution before I move ahead with the PR.

amitalokbera avatar May 29 '25 17:05 amitalokbera

Sounds good @amitalokbera! I might even take it one step further (not sure how easy / hard it will be) - have a module per LLM. So you can even just install traceloop-sdk[openai] - which is a common scenario imo. Same for frameworks - people are more likely to use just one framework, and not all of them.

nirga avatar May 30 '25 01:05 nirga

Hi @nirga I want to take this up and soon will raise a PR

kusumanchisrikanth29 avatar Jun 13 '25 07:06 kusumanchisrikanth29

Hey @nirga, sorry for the super late update, I was in the middle of a job switch and didn’t get a chance to work on this earlier.

I just want to confirm the expected behavior for optional dependencies:

  • If someone wants to install only specific extras, like openai and chromadb, the correct command would be:

    pip install "traceloop-sdk[openai,chromadb]"

  • If a user wants to install all optional dependencies, they can use: pip install "traceloop-sdk[all]"

amitalokbera avatar Jun 22 '25 10:06 amitalokbera

Hi! I'm Madesh. I'm new to open source and would love to contribute. This looks like a good first issue β€” may I take it up?

madesh6554 avatar Jun 24 '25 16:06 madesh6554

Hi! I'm a beginner looking to contribute through GSoC. Can I take this issue? Also, could you guide me on how you'd prefer the installation behavior to be optimized (e.g. optional dependencies, extras)?

Chetan27Pareek avatar Jun 27 '25 16:06 Chetan27Pareek

Hi! I am a beginner to open source and would love to contribute. I believe optional dependencies makes sense here, but I think it might make the setup more complex.

Sidharthtech avatar Jul 18 '25 16:07 Sidharthtech

Hi, I'd like to help implement this optional dependencies feature! Please assign me or let me know if there are tips for new contributors.

DHANUSHRAJA22 avatar Aug 25 '25 19:08 DHANUSHRAJA22

hi , i want to work on this issue , if i solve it will u merge ? @InAnYan

harsha08-2k6 avatar Oct 29 '25 06:10 harsha08-2k6