Integrate with Open-WebUI

[parthpat12]

Can this be used as "Pipe" in Open-WebUI front?

[thiswillbeyourgithub]

Hi, this is definitely the plan. But the issue is that you can't have external dependencies in a pipe or in a function, so it has to be a pipeline. Or another possibility would be to make my project fit inside a Docker container and expose an API that could be accessed as a tool from an LLM.

Either way, it shouldn't be too difficult but would probably be best done after I reworked some parts of the code to enhance the UI of this project. Right now the workflow for CLI is pretty awkward in my opinion. One additional benefit would be to wait for pipelines and open WebUI to become more stable.

I have to say I'm quite bullish on the pipelines overall.

If for some reason you are in a hurry, you could make it work quite quickly by taking a look at my examples. Maybe the ntfy one. Then write a fastAPI endpoint. Then add a loop. Then write a open WebUI function that calls this endpoint. I'm thinking both should be under 200 lines of code combined.

Interested?

[thiswillbeyourgithub]

Hello again. I just realized that adding the wdoc summary prompt as a system message to an open-webui model equipped with the tools web scrape and youtube transcript provider is already pretty nice! You can create a chat, put a link and have a reasonnably well formatted summary.

If anyone has been using wdoc with open-webui I'm all ears for other tricks.

[thiswillbeyourgithub]

Good news : I recently found out that we can supply a requirements to install in openwebui tools (which are much simpler than pipelines). I don't know if this is stable or a bug because it's not documented and the lack of requirements is supposed to be a defining distinction between tools and pipelines so in the meantime I don't want to code a tool befire getting an official reply.

I asked one of the people involved here:

Hi @justinh-rahb I noticed yesterday that actually we can define requirements in tools and they are pip installed. Is this something not yet documented or not yet removed? I'm planning on using it to import wdoc, my sophisticated rag setup that does pretty darn good summaries but want to know first if requirements in tools is stable as I don't have the time to familiarize myself with pipelines.

The exact code that worked is this:

"""
---
requirements: wdoc==2.5.7
---
"""

My opinion: now that we have permissions, we can forbid users from adding tools so it makes sense to allow requirements and would VASTLY increase what the community could do. Personnaly i would be in favor of a special right needed to use pip, meaning we can authorize a user to use pip install + use requirements, or to only use requirements if they are already pip installed by an admin.

And while i'm at it:

• the code cannot handle substituting """ by ''' which seems like a bug to me.
• i think I saw it was using pip instead of uv. Uv is way faster.
• once a tool is removed or a requirement no longer mentionned it would be nice to uv pip remove what was installed, to avoid clogging up the install.

Let me know if you want me to open up an issue for any of the points I raised or mention them in #5486

Of course if anyone wants to do it be my guest. It should be pretty simple and allow to do wdoc summaries very simply. For querying a but more work and for adding documents even more work but overall not that hard.

I still think it would be nice to have a cleaner python api beforehand though.

Edit: it lools like the initial commits were done in august 2024 so looks like it's a feature here to stay

[thiswillbeyourgithub]

Still a work in progress but here is the tool : https://github.com/thiswillbeyourgithub/openwebui_custom_pipes_filters/blob/main/tools/wdoc_tools.py

The parser seems to work okay. The summarizer not yet because open-webui's logging is broken at the moment.

I quite like it and plan to make it a full frontend for open-webui. Plus it's just a tool and does not need to setup pipelines.

[layfellow]

I fixed the summarizer.

Submitted a pull request: https://github.com/thiswillbeyourgithub/openwebui_custom_pipes_filters/pull/2

[thiswillbeyourgithub]

I pushed a new version I have not yet tested if you want. Largely revamped following my new ideas. It is using the latest wdoc release I just pushed although would work with previous too.

[layfellow]

Cool! Will update my wdoc and test it.

[thiswillbeyourgithub]

Did try some more things. I can't for the life of me figure out why it keep crashing because of lack of ANTHROPIC_API_KEY. It is as if the input arguments or env variables are ignored or something. The thing is the latest open-webui release broke the logs so I struggle to see what's going on. And trying things with importlib surprisingly does not even help.

Unless you have ideas I think I'll put on hold until open-webui fixes the logger.

For reference: I don't use any UserValves and here are my Valves values:

{"embed_model": "openai/text-embedding-3-large", "model": "openrouter/anthropic/claude-3.7-sonnet", "query_eval_model": "openrouter/google/gemini-2.0-flash-001"}

{"embed_model": "openai/text-embedding-3-large", "model": "openrouter/anthropic/claude-3.7-sonnet", "query_eval_model": "openrouter/google/gemini-2.0-flash-001"}

{"WDOC_LITELLM_USER": "$USER", "WDOC_LITELLM_TAGS": "open-webui", "WDOC_STRICT_DOCDICT": "False", "WDOC_TYPECHECKING": "crash", "WDOC_STRICT_DOCDICT": "true", "WDOC_DEBUGGER": "true", "WDOC_EXPIRE_CACHE_DAYS": "90", "WDOC_IMPORT_TYPE": "thread", "WDOC_DEFAULT_MODEL": "openrouter/anthropic/claude-3.7-sonnet", "WDOC_DEFAULT_EMBED_MODEL": "openai/text-embedding-3-small", "WDOC_DEFAULT_QUERY_EVAL_MODEL": "openrouter/google/gemini-2.0-flash-001"}

[thiswillbeyourgithub]

I'm running into this issue: Couldn't import wdoc: '/usr/local/lib/python3.11/site-packages/torchaudio/lib/libtorchaudio.so: undefined symbol: _ZN2at4_ops9fft_irfft4callERKNS_6TensorESt8optionalIN3c106SymIntEElS5_ISt17basic_string_viewIcSt11char_traitsIcEEE'. Even though I'm able to do import wdoc in the python repl inside the container. I tried to upgrade or reinstall torchaudio and torch to no avail.

Any idea is much appreciated.

[thiswillbeyourgithub]

This seems to be due to this env variable set in openwebui: "LD_LIBRARY_PATH": "/usr/local/lib/python3.11/site-packages/cv2/../../lib64::/usr/local/lib/python3.11/site-packages/torch/lib:/usr/local/lib/python3.11/site-packages/nvidia/cudnn/lib"

[thiswillbeyourgithub]

progress: it seems that "os.environ["WDOC_IMPORT_TYPE"] = "native"" helps. I also added a try block to avoid crashing if torchaudio fails to be imported (it can be tricky to install anyway).

Now I just need to understand why the arguments are not properly passed (it is still asking me for my anthropic api key)

[thiswillbeyourgithub]

Figured out most of the issues now. Still some kinks to work out but a working version seems pretty close now.

[thiswillbeyourgithub]

Well it seems to be working now! At least as first demo. I'm closing this now but your help in testing this thing would be much appreciated. For further developments please consider making an issue on this repo instead.

edit: finally pushed here: https://openwebui.com/t/qqqqqqqqqqqqqqqqqqqq/wdoctool