Conflict between Pipelines' Python environment dependencies and custom tool dependencies

[yueguobin]

I am encountering a dependency conflict between the Python environment used by Pipelines and the dependencies required by the custom tools I want to use, specifically with the Pydantic library.

• Langchain version 0.3.21 requires pydantic<3.0.0,>=2.7.4.
• Pipelines, however, uses pydantic==2.7.1.

When I specify a custom Python environment in my Python file for the tool to run in, Pipelines doesn't seem to respect this environment and continues using its default environment. This means that the dependencies required by Langchain and other custom tools cannot be used properly within Pipelines, as Pipelines and the tools cannot share the same Python environment.

This limitation hinders the ability to extend applications, as it’s impossible to resolve the dependency conflicts between Pipelines’ requirements and the libraries required by other tools.

Is there a way to either:

• Allow Pipelines to respect custom Python environments for running agents or tools,
• Or, provide a method to install custom dependencies without breaking Pipelines' own environment?

This would greatly enhance the flexibility and extensibility of Pipelines.

Thank you for your attention to this issue.

[phiwan-dev]

Here is a hacky workaround: I think the pipelines pydantic version requirement is not that strict. For me it works with 2.10.6 as well.

[SamuelBortolinAfliant]

I would love to have less restricting requirements versions, I have issues in installing latest versions of Ollama and llamaindex.

[Maleya]

I think the real issue here is that there's no containerization or virtual environment separation for individual pipeline requirements. I am going to if theres an existing issue for this