storm icon indicating copy to clipboard operation
storm copied to clipboard
Published 3 months ago • verified publisher icon stanford-oval

Want to run fully locally using OLLAMA and SEARXNG

Open Shyamnath-Sankar opened this issue 1 year ago • 6 comments

can anyone please guide me to create this fully locally in my pc

llm as OLLAMA CLIENT web search searXNG

i will be very must greatful if you help me through

Shyamnath-Sankar avatar Oct 14 '24 10:10 Shyamnath-Sankar

Hi, have you tried the example script here: https://github.com/stanford-oval/storm/blob/main/examples/storm_examples/run_storm_wiki_ollama_with_searxng.py

Yucheng-Jiang avatar Oct 19 '24 18:10 Yucheng-Jiang

What is exactly the searxng_api_url of Searxng? Is it just the root domain, or /search?q= ? Because mine results in unsuccessful outcomes:

python run_storm_wiki_ollama_with_searxng.py --searxng-api-url http://localhost:MY_PORT --do-research --do-generate-outline Topic: dragons root : ERROR : Error occurs when searching query : Expecting value: line 1 column 1 (char 0)

Searxng works elsewhere (e.g. Open-Webui). Any ideas what might be wrong?

cl93a avatar Jan 01 '25 20:01 cl93a 

python examples/storm_examples/run_storm_wiki_ollama_with_searxng.py --model=hf.co/bartowski/Llama-3.3-70B-Instruct-GGUF:IQ4_XS --searxng-api-url=http://localhost:MY_PORT/search? --do-research --do-generate-outline --do-generate-article --do-polish-article --retriever=searxng

This seems to work for me ~50% of the time. And gives me errors like this the other half:

Error: CODE 500 - {"error":"POST predict: Post \"http://127.0.0.1:33643/completion\": EOF"}
root : ERROR    : Error occurs when generating answer: can only join an iterable
Error: CODE 500 - {"error":"POST predict: Post \"http://127.0.0.1:33643/completion\": EOF"}
Error: CODE 500 - {"error":"an unknown error was encountered while running the model "}
Error: CODE 500 - {"error":"POST predict: Post \"http://127.0.0.1:45537/completion\": EOF"}
root : ERROR    : Error occurs when generating answer: can only join an iterable
Traceback (most recent call last):
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/urllib3/connectionpool.py", line 534, in _make_request
    response = conn.getresponse()
               ^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/urllib3/connection.py", line 516, in getresponse
    httplib_response = super().getresponse()
                       ^^^^^^^^^^^^^^^^^^^^^
  File ".../anaconda3/lib/python3.11/http/client.py", line 1386, in getresponse
    response.begin()
  File ".../anaconda3/lib/python3.11/http/client.py", line 325, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File ".../anaconda3/lib/python3.11/http/client.py", line 286, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../anaconda3/lib/python3.11/socket.py", line 706, in readinto
    return self._sock.recv_into(b)
           ^^^^^^^^^^^^^^^^^^^^^^^
TimeoutError: timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/requests/adapters.py", line 667, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/urllib3/connectionpool.py", line 841, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/urllib3/util/retry.py", line 474, in increment
    raise reraise(type(error), error, _stacktrace)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/urllib3/util/util.py", line 39, in reraise
    raise value
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/urllib3/connectionpool.py", line 536, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/urllib3/connectionpool.py", line 367, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='localhost', port=11434): Read timed out. (read timeout=120)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".../soft/storm/examples/storm_examples/run_storm_wiki_ollama_with_searxng.py", line 154, in <module>
    main(parser.parse_args())
  File ".../soft/storm/examples/storm_examples/run_storm_wiki_ollama_with_searxng.py", line 101, in main
    runner.run(
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/knowledge_storm/storm_wiki/engine.py", line 394, in run
    information_table = self.run_knowledge_curation_module(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/knowledge_storm/interface.py", line 499, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/knowledge_storm/storm_wiki/engine.py", line 219, in run_knowledge_curation_module
    self.storm_knowledge_curation_module.research(
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/knowledge_storm/storm_wiki/modules/knowledge_curation.py", line 379, in research
    conversations = self._run_conversation(
                    ^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/knowledge_storm/storm_wiki/modules/knowledge_curation.py", line 340, in _run_conversation
    conv = future.result()
           ^^^^^^^^^^^^^^^
  File ".../anaconda3/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File ".../anaconda3/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File ".../anaconda3/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/knowledge_storm/storm_wiki/modules/knowledge_curation.py", line 318, in run_conv
    return conv_simulator(
           ^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/dspy/primitives/program.py", line 26, in __call__
    return self.forward(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/knowledge_storm/storm_wiki/modules/knowledge_curation.py", line 61, in forward
    user_utterance = self.wiki_writer(
                     ^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/dspy/primitives/program.py", line 26, in __call__
    return self.forward(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/knowledge_storm/storm_wiki/modules/knowledge_curation.py", line 117, in forward
    question = self.ask_question_with_persona(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/dspy/predict/predict.py", line 61, in __call__
    return self.forward(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/dspy/predict/chain_of_thought.py", line 59, in forward
    return super().forward(signature=signature, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/dspy/predict/predict.py", line 103, in forward
    x, C = dsp.generate(template, **config)(x, stage=self.stage)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/dsp/primitives/predict.py", line 77, in do_generate
    completions: list[dict[str, Any]] = generator(prompt, **kwargs)
                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/dsp/modules/ollama.py", line 171, in __call__
    response = self.request(prompt, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/dsp/modules/ollama.py", line 145, in request
    return self.basic_request(prompt, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/dsp/modules/ollama.py", line 97, in basic_request
    response = requests.post(urlstr, json=settings_dict, timeout=self.timeout_s)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.local/share/virtualenvs/storm-CTT7sfkk/lib/python3.11/site-packages/requests/adapters.py", line 713, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPConnectionPool(host='localhost', port=11434): Read timed out. (read timeout=120)

gsoul avatar Jan 04 '25 22:01 gsoul

ok, my bug seem to be fixed by updating ollama. The url for searxng is "http://localhost:MY_PORT/search? "

gsoul avatar Jan 05 '25 15:01 gsoul

Had very similar issues - ended up to be missing entry in searXNG settings.yml (added -json to formats). Mitigated all root: ERRORs

adibu avatar Jan 08 '25 17:01 adibu

For those running their own searxng instance here is a list of things I had to do to get storm working (some what, still see some errors though)

  • Format the URL correctly: --searxng-api-url 'https://mydomain.com/search?'
  • I was getting errors when storm was running similar to root : ERROR : Error occurs when searching query : 'results' and I needed to disable some search engines either due to too many requests or CAPTCHAS. Add this to your settings.yaml in searxng
engines:
  - name: duckduckgo
    disabled: true
  - name: qwant
    disabled: true
  - name: brave
    disabled: true
  • Add json to formats in settings.yaml
search:
  formats:
    - html
    - json

tuxthepenguin84 avatar Feb 21 '25 21:02 tuxthepenguin84