Perplexica
Perplexica copied to clipboard
Published
20 hours ago •
ItzCrazyKns
ItzCrazyKns
FileNotFoundError for missing 'framalibre.py' in searxng engine initialization
Describe the bug The application is experiencing multiple startup errors during the initialization of specific search engine modules, particularly "framalibre". The log files also indicate missing configuration files, namely limiter.toml for bot detection settings which may lead to further issues if not addressed.
To Reproduce
docker compose up -d
Expected behavior The expected behavior includes successful initialization of all search engines configured for the service, including "framalibre". There should not be errors pertaining to missing modules or configuration files. The system should ideally handle missing configurations gracefully or prompt the user/administrator to rectify missing data.
Additional context The errors regarding the "framalibre" search engine suggest either an issue with its configuration pointing to a non-existent module or perhaps the module was set to be included without being properly implemented or deployed. Additionally, the recurring warnings about the missing limiter.toml could imply a missing or incorrect path setting or failed configurations, which could affect the bot detection efficiency and may allow harmful traffic that can potentially cause service disruptions.
perplexica-backend-1 | yarn run v1.22.19
perplexica-backend-1 | $ node dist/app.js
perplexica-backend-1 | WebSocket server started on port 3001
perplexica-backend-1 | (node:28) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
perplexica-backend-1 | (Use `node --trace-deprecation ...` to show where the warning was created)
perplexica-backend-1 | API server started on port 3001
perplexica-backend-1 | Connection closed
perplexica-backend-1 | Connection closed
perplexica-frontend-1 | yarn run v1.22.19
perplexica-frontend-1 | $ next start
perplexica-frontend-1 | ▲ Next.js 14.1.4
perplexica-frontend-1 | - Local: http://localhost:3000
perplexica-frontend-1 |
perplexica-frontend-1 | ✓ Ready in 434ms
searxng-1 | SearXNG version 2024.4.28+e6c345f89
searxng-1 | Create /etc/searxng/uwsgi.ini
searxng-1 | Use existing /etc/searxng/settings.yml
searxng-1 | Listen on 0.0.0.0:8080
searxng-1 | [uWSGI] getting INI configuration from /etc/searxng/uwsgi.ini
searxng-1 | [uwsgi-static] added mapping for /static => /usr/local/searxng/searx/static
searxng-1 | *** Starting uWSGI 2.0.23 (64bit) on [Sun Apr 28 12:33:06 2024] ***
searxng-1 | compiled with version: 13.2.1 20231014 on 30 November 2023 14:34:33
searxng-1 | os: Linux-6.6.22-linuxkit #1 SMP Fri Mar 29 12:21:27 UTC 2024
searxng-1 | nodename: ed72579ef066
searxng-1 | machine: aarch64
searxng-1 | clock source: unix
searxng-1 | pcre jit disabled
searxng-1 | detected number of CPU cores: 12
searxng-1 | current working directory: /usr/local/searxng
searxng-1 | detected binary path: /usr/sbin/uwsgi
searxng-1 | chdir() to /usr/local/searxng/searx/
searxng-1 | your memory page size is 4096 bytes
searxng-1 | detected max file descriptor number: 1048576
searxng-1 | building mime-types dictionary from file /etc/mime.types...1390 entry found
searxng-1 | lock engine: pthread robust mutexes
searxng-1 | thunder lock: disabled (you can enable it with --thunder-lock)
searxng-1 | uwsgi socket 0 bound to TCP address 0.0.0.0:8080 fd 3
searxng-1 | Python version: 3.11.9 (main, Apr 14 2024, 13:40:00) [GCC 13.2.1 20231014]
searxng-1 | Python main interpreter initialized at 0xffffaaa10ac0
searxng-1 | python threads support enabled
searxng-1 | your server socket listen backlog is limited to 100 connections
searxng-1 | your mercy for graceful operations on workers is 60 seconds
searxng-1 | mapped 1568736 bytes (1531 KB) for 48 cores
searxng-1 | *** Operational MODE: preforking+threaded ***
searxng-1 | added /usr/local/searxng/ to pythonpath.
searxng-1 | spawned uWSGI master process (pid: 7)
searxng-1 | spawned uWSGI worker 1 (pid: 12, cores: 4)
searxng-1 | spawned uWSGI worker 2 (pid: 14, cores: 4)
searxng-1 | spawned 4 offload threads for uWSGI worker 1
searxng-1 | spawned uWSGI worker 3 (pid: 19, cores: 4)
searxng-1 | spawned 4 offload threads for uWSGI worker 2
searxng-1 | spawned uWSGI worker 4 (pid: 23, cores: 4)
searxng-1 | spawned 4 offload threads for uWSGI worker 3
searxng-1 | spawned uWSGI worker 5 (pid: 28, cores: 4)
searxng-1 | spawned uWSGI worker 6 (pid: 29, cores: 4)
searxng-1 | spawned uWSGI worker 7 (pid: 35, cores: 4)
searxng-1 | spawned 4 offload threads for uWSGI worker 4
searxng-1 | spawned 4 offload threads for uWSGI worker 5
searxng-1 | spawned uWSGI worker 8 (pid: 42, cores: 4)
searxng-1 | spawned uWSGI worker 9 (pid: 46, cores: 4)
searxng-1 | spawned 4 offload threads for uWSGI worker 7
searxng-1 | spawned 4 offload threads for uWSGI worker 6
searxng-1 | spawned uWSGI worker 10 (pid: 54, cores: 4)
searxng-1 | spawned 4 offload threads for uWSGI worker 8
searxng-1 | spawned 4 offload threads for uWSGI worker 9
searxng-1 | spawned uWSGI worker 11 (pid: 58, cores: 4)
searxng-1 | spawned uWSGI worker 12 (pid: 59, cores: 4)
searxng-1 | spawned 4 offload threads for uWSGI worker 10
searxng-1 | spawned 4 offload threads for uWSGI worker 11
searxng-1 | spawned 4 offload threads for uWSGI worker 12
searxng-1 | 2024-04-28 12:33:07,066 ERROR:searx.engines: Cannot load engine "framalibre"
searxng-1 | Traceback (most recent call last):
searxng-1 | File "/usr/local/searxng/searx/engines/__init__.py", line 122, in load_engine
searxng-1 | engine = load_module(module_name + '.py', ENGINE_DIR)
searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
searxng-1 | File "/usr/local/searxng/searx/utils.py", line 428, in load_module
searxng-1 | spec.loader.exec_module(module)
searxng-1 | File "<frozen importlib._bootstrap_external>", line 936, in exec_module
searxng-1 | File "<frozen importlib._bootstrap_external>", line 1073, in get_code
searxng-1 | File "<frozen importlib._bootstrap_external>", line 1130, in get_data
searxng-1 | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/searxng/searx/engines/framalibre.py'
searxng-1 | 2024-04-28 12:33:07,066 ERROR:searx.engines: Cannot load engine "framalibre"
searxng-1 | Traceback (most recent call last):
searxng-1 | File "/usr/local/searxng/searx/engines/__init__.py", line 122, in load_engine
searxng-1 | engine = load_module(module_name + '.py', ENGINE_DIR)
searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
searxng-1 | File "/usr/local/searxng/searx/utils.py", line 428, in load_module
searxng-1 | spec.loader.exec_module(module)
searxng-1 | File "<frozen importlib._bootstrap_external>", line 936, in exec_module
searxng-1 | File "<frozen importlib._bootstrap_external>", line 1073, in get_code
searxng-1 | File "<frozen importlib._bootstrap_external>", line 1130, in get_data
searxng-1 | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/searxng/searx/engines/framalibre.py'
searxng-1 | 2024-04-28 12:33:07,067 ERROR:searx.engines: Cannot load engine "framalibre"
searxng-1 | Traceback (most recent call last):
searxng-1 | File "/usr/local/searxng/searx/engines/__init__.py", line 122, in load_engine
searxng-1 | engine = load_module(module_name + '.py', ENGINE_DIR)
searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
searxng-1 | File "/usr/local/searxng/searx/utils.py", line 428, in load_module
searxng-1 | spec.loader.exec_module(module)
searxng-1 | File "<frozen importlib._bootstrap_external>", line 936, in exec_module
searxng-1 | File "<frozen importlib._bootstrap_external>", line 1073, in get_code
searxng-1 | File "<frozen importlib._bootstrap_external>", line 1130, in get_data
searxng-1 | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/searxng/searx/engines/framalibre.py'
searxng-1 | 2024-04-28 12:33:07,066 ERROR:searx.engines: Cannot load engine "framalibre"
searxng-1 | Traceback (most recent call last):
searxng-1 | File "/usr/local/searxng/searx/engines/__init__.py", line 122, in load_engine
searxng-1 | engine = load_module(module_name + '.py', ENGINE_DIR)
searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
searxng-1 | File "/usr/local/searxng/searx/utils.py", line 428, in load_module
searxng-1 | spec.loader.exec_module(module)
searxng-1 | File "<frozen importlib._bootstrap_external>", line 936, in exec_module
searxng-1 | File "<frozen importlib._bootstrap_external>", line 1073, in get_code
searxng-1 | File "<frozen importlib._bootstrap_external>", line 1130, in get_data
searxng-1 | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/searxng/searx/engines/framalibre.py'
searxng-1 | 2024-04-28 12:33:07,067 ERROR:searx.engines: Cannot load engine "framalibre"
searxng-1 | Traceback (most recent call last):
searxng-1 | File "/usr/local/searxng/searx/engines/__init__.py", line 122, in load_engine
searxng-1 | engine = load_module(module_name + '.py', ENGINE_DIR)
searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
searxng-1 | File "/usr/local/searxng/searx/utils.py", line 428, in load_module
searxng-1 | spec.loader.exec_module(module)
searxng-1 | File "<frozen importlib._bootstrap_external>", line 936, in exec_module
searxng-1 | File "<frozen importlib._bootstrap_external>", line 1073, in get_code
searxng-1 | File "<frozen importlib._bootstrap_external>", line 1130, in get_data
searxng-1 | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/searxng/searx/engines/framalibre.py'
searxng-1 | 2024-04-28 12:33:07,067 ERROR:searx.engines: Cannot load engine "framalibre"
searxng-1 | Traceback (most recent call last):
searxng-1 | File "/usr/local/searxng/searx/engines/__init__.py", line 122, in load_engine
searxng-1 | engine = load_module(module_name + '.py', ENGINE_DIR)
searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
searxng-1 | File "/usr/local/searxng/searx/utils.py", line 428, in load_module
searxng-1 | spec.loader.exec_module(module)
searxng-1 | File "<frozen importlib._bootstrap_external>", line 936, in exec_module
searxng-1 | File "<frozen importlib._bootstrap_external>", line 1073, in get_code
searxng-1 | File "<frozen importlib._bootstrap_external>", line 1130, in get_data
searxng-1 | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/searxng/searx/engines/framalibre.py'
searxng-1 | 2024-04-28 12:33:07,067 ERROR:searx.engines: Cannot load engine "framalibre"
searxng-1 | Traceback (most recent call last):
searxng-1 | File "/usr/local/searxng/searx/engines/__init__.py", line 122, in load_engine
searxng-1 | engine = load_module(module_name + '.py', ENGINE_DIR)
searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
searxng-1 | File "/usr/local/searxng/searx/utils.py", line 428, in load_module
searxng-1 | spec.loader.exec_module(module)
searxng-1 | File "<frozen importlib._bootstrap_external>", line 936, in exec_module
searxng-1 | File "<frozen importlib._bootstrap_external>", line 1073, in get_code
searxng-1 | File "<frozen importlib._bootstrap_external>", line 1130, in get_data
searxng-1 | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/searxng/searx/engines/framalibre.py'
searxng-1 | 2024-04-28 12:33:07,067 ERROR:searx.engines: Cannot load engine "framalibre"
searxng-1 | Traceback (most recent call last):
searxng-1 | File "/usr/local/searxng/searx/engines/__init__.py", line 122, in load_engine
searxng-1 | engine = load_module(module_name + '.py', ENGINE_DIR)
searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
searxng-1 | File "/usr/local/searxng/searx/utils.py", line 428, in load_module
searxng-1 | spec.loader.exec_module(module)
searxng-1 | File "<frozen importlib._bootstrap_external>", line 936, in exec_module
searxng-1 | File "<frozen importlib._bootstrap_external>", line 1073, in get_code
searxng-1 | File "<frozen importlib._bootstrap_external>", line 1130, in get_data
searxng-1 | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/searxng/searx/engines/framalibre.py'
searxng-1 | 2024-04-28 12:33:07,067 ERROR:searx.engines: Cannot load engine "framalibre"
searxng-1 | Traceback (most recent call last):
searxng-1 | File "/usr/local/searxng/searx/engines/__init__.py", line 122, in load_engine
searxng-1 | engine = load_module(module_name + '.py', ENGINE_DIR)
searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
searxng-1 | File "/usr/local/searxng/searx/utils.py", line 428, in load_module
searxng-1 | spec.loader.exec_module(module)
searxng-1 | File "<frozen importlib._bootstrap_external>", line 936, in exec_module
searxng-1 | File "<frozen importlib._bootstrap_external>", line 1073, in get_code
searxng-1 | File "<frozen importlib._bootstrap_external>", line 1130, in get_data
searxng-1 | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/searxng/searx/engines/framalibre.py'
searxng-1 | 2024-04-28 12:33:07,067 ERROR:searx.engines: Cannot load engine "framalibre"
searxng-1 | Traceback (most recent call last):
searxng-1 | File "/usr/local/searxng/searx/engines/__init__.py", line 122, in load_engine
searxng-1 | engine = load_module(module_name + '.py', ENGINE_DIR)
searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
searxng-1 | File "/usr/local/searxng/searx/utils.py", line 428, in load_module
searxng-1 | spec.loader.exec_module(module)
searxng-1 | File "<frozen importlib._bootstrap_external>", line 936, in exec_module
searxng-1 | File "<frozen importlib._bootstrap_external>", line 1073, in get_code
searxng-1 | File "<frozen importlib._bootstrap_external>", line 1130, in get_data
searxng-1 | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/searxng/searx/engines/framalibre.py'
searxng-1 | 2024-04-28 12:33:07,068 ERROR:searx.engines: Cannot load engine "framalibre"
searxng-1 | Traceback (most recent call last):
searxng-1 | File "/usr/local/searxng/searx/engines/__init__.py", line 122, in load_engine
searxng-1 | engine = load_module(module_name + '.py', ENGINE_DIR)
searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
searxng-1 | File "/usr/local/searxng/searx/utils.py", line 428, in load_module
searxng-1 | spec.loader.exec_module(module)
searxng-1 | File "<frozen importlib._bootstrap_external>", line 936, in exec_module
searxng-1 | File "<frozen importlib._bootstrap_external>", line 1073, in get_code
searxng-1 | File "<frozen importlib._bootstrap_external>", line 1130, in get_data
searxng-1 | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/searxng/searx/engines/framalibre.py'
searxng-1 | 2024-04-28 12:33:07,066 ERROR:searx.engines: Cannot load engine "framalibre"
searxng-1 | Traceback (most recent call last):
searxng-1 | File "/usr/local/searxng/searx/engines/__init__.py", line 122, in load_engine
searxng-1 | engine = load_module(module_name + '.py', ENGINE_DIR)
searxng-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
searxng-1 | File "/usr/local/searxng/searx/utils.py", line 428, in load_module
searxng-1 | spec.loader.exec_module(module)
searxng-1 | File "<frozen importlib._bootstrap_external>", line 936, in exec_module
searxng-1 | File "<frozen importlib._bootstrap_external>", line 1073, in get_code
searxng-1 | File "<frozen importlib._bootstrap_external>", line 1130, in get_data
searxng-1 | FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/searxng/searx/engines/framalibre.py'
searxng-1 | 2024-04-28 12:33:07,102 WARNING:searx.botdetection.config: missing config file: /etc/searxng/limiter.toml
searxng-1 | 2024-04-28 12:33:07,103 WARNING:searx.botdetection.config: missing config file: /etc/searxng/limiter.toml
searxng-1 | WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0xffffaaa10ac0 pid: 29 (default app)
searxng-1 | WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0xffffaaa10ac0 pid: 14 (default app)
searxng-1 | 2024-04-28 12:33:07,105 WARNING:searx.botdetection.config: missing config file: /etc/searxng/limiter.toml
searxng-1 | WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0xffffaaa10ac0 pid: 42 (default app)
searxng-1 | 2024-04-28 12:33:07,105 WARNING:searx.botdetection.config: missing config file: /etc/searxng/limiter.toml
searxng-1 | 2024-04-28 12:33:07,105 WARNING:searx.botdetection.config: missing config file: /etc/searxng/limiter.toml
searxng-1 | WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0xffffaaa10ac0 pid: 59 (default app)
searxng-1 | WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0xffffaaa10ac0 pid: 12 (default app)
searxng-1 | 2024-04-28 12:33:07,112 WARNING:searx.botdetection.config: missing config file: /etc/searxng/limiter.toml
searxng-1 | WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0xffffaaa10ac0 pid: 28 (default app)
searxng-1 | 2024-04-28 12:33:07,113 WARNING:searx.botdetection.config: missing config file: /etc/searxng/limiter.toml
searxng-1 | WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0xffffaaa10ac0 pid: 35 (default app)
searxng-1 | 2024-04-28 12:33:07,114 WARNING:searx.botdetection.config: missing config file: /etc/searxng/limiter.toml
searxng-1 | 2024-04-28 12:33:07,113 WARNING:searx.botdetection.config: missing config file: /etc/searxng/limiter.toml
searxng-1 | 2024-04-28 12:33:07,114 WARNING:searx.botdetection.config: missing config file: /etc/searxng/limiter.toml
searxng-1 | WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0xffffaaa10ac0 pid: 46 (default app)
searxng-1 | WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0xffffaaa10ac0 pid: 23 (default app)
searxng-1 | WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0xffffaaa10ac0 pid: 19 (default app)
searxng-1 | 2024-04-28 12:33:07,114 WARNING:searx.botdetection.config: missing config file: /etc/searxng/limiter.toml
searxng-1 | WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0xffffaaa10ac0 pid: 54 (default app)
searxng-1 | 2024-04-28 12:33:07,118 WARNING:searx.botdetection.config: missing config file: /etc/searxng/limiter.toml
searxng-1 | WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0xffffaaa10ac0 pid: 58 (default app)
Seems like SearxNG removed the framalibre engine, it is present in our SearxNG settings file so that might be causing an issue.
I reproduced the issue and I see that SearxNG works completely fine, it is not affected by that error. Please ignore any errors (except for something major) from SearxNG. If you face any issues, feel free to re-open this issue. Thanks
It might not be related to SearxNG. I can open http://localhost:3000/, but when I search for something, it gets stuck at loading. The browser console states, "WebSocket is already in CLOSING or CLOSED state."
Below are the Docker logs for perplexica-perplexica-backend-1:
2024-04-28 09:26:32 yarn run v1.22.19
2024-04-28 09:26:32 $ node dist/app.js
2024-04-28 09:26:33 WebSocket server started on port 3001
2024-04-28 09:26:33 API server started on port 3001
2024-04-28 09:26:37 Connection closed
2024-04-28 09:26:33 (node:28) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
2024-04-28 09:26:33 (Use `node --trace-deprecation ...` to show where the warning was created)
2024-04-28 09:28:01 Connection closed
It might be related to port 3001 on my Apple M3.
Try refreshing your page. If that doesn't work provide me with more context so I can help.
Additionally, it can be related to Ollama, if you don't use the right model name. You can update the model to use from the settings dialog.