pyspider
pyspider copied to clipboard
Error to start webui service
Environment
- pip version: 8.1.1
- Python version: 2.7
- OS: Ubuntu 16.04
- Redis: 4.0.11
- MongoDB: 3.6.7
Description When I start webui, and I see the error logs. So please help me to fix it. Many thanks
Output logs
nohup: ignoring input
[W 180824 13:26:04 __init__:67] redis DB must zero-based numeric index, using 0 instead
[W 180824 13:26:04 __init__:67] redis DB must zero-based numeric index, using 0 instead
[W 180824 13:26:04 __init__:67] redis DB must zero-based numeric index, using 0 instead
[W 180824 13:26:04 __init__:67] redis DB must zero-based numeric index, using 0 instead
[W 180824 13:26:04 __init__:67] redis DB must zero-based numeric index, using 0 instead
Traceback (most recent call last):
File "/usr/local/bin/pyspider", line 9, in <module>
load_entry_point('pyspider==0.3.10', 'console_scripts', 'pyspider')()
File "/usr/local/lib/python2.7/dist-packages/pyspider-0.3.10-py2.7.egg/pyspider/run.py", line 754, in main
cli()
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/pyspider-0.3.10-py2.7.egg/pyspider/run.py", line 384, in webui
app.run(host=host, port=port)
File "/usr/local/lib/python2.7/dist-packages/pyspider-0.3.10-py2.7.egg/pyspider/webui/app.py", line 59, in run
from .webdav import dav_app
File "/usr/local/lib/python2.7/dist-packages/pyspider-0.3.10-py2.7.egg/pyspider/webui/webdav.py", line 216, in <module>
dav_app = WsgiDAVApp(config)
File "/usr/local/lib/python2.7/dist-packages/WsgiDAV-3.0.0a3-py2.7.egg/wsgidav/wsgidav_app.py", line 128, in __init__
_check_config(config)
File "/usr/local/lib/python2.7/dist-packages/WsgiDAV-3.0.0a3-py2.7.egg/wsgidav/wsgidav_app.py", line 111, in _check_config
raise ValueError("Invalid configuration:\n - " + "\n - ".join(errors))
ValueError: Invalid configuration:
- Deprecated option 'domaincontroller': use 'domain_controller' instead.
- Deprecated option 'dir_browser.enable': use 'middleware_stack' instead.
WsgiDAV released a new version -- pre-release 3.x 'dir_browser.enable' DEPRECATED with 2.4.0 and 'domaincontroller' updated to 'http_authenticator.HTTPAuthenticator'
reference:https://wsgidav.readthedocs.io/en/latest/addons-ntdc.html
You can change pyspider/webui/webdav.py line 203:
config = DEFAULT_CONFIG.copy()
config.update({
'mount_path': '/dav',
'provider_mapping': {
'/': ScriptProvider(app)
},
#'domaincontroller': NeedAuthController(app),
'http_authenticator': {
'HTTPAuthenticator':NeedAuthController(app),
},
'verbose': 1 if app.debug else 0,
'dir_browser': {'davmount': False,
#'enable': True,
'msmount': False,
'response_trailer': ''},
})
dav_app = WsgiDAVApp(config)
Last rebuild #python setup.py install
or You can try editing the version of wsgidav in ./setup.py
I tried the idea but the problem still on. centos7 python2.7
error log:
[root@dn32 pyspider]# pyspider
[W 181123 11:37:53 run:413] phantomjs not found, continue running without it.
[I 181123 11:37:55 result_worker:49] result_worker starting...
[I 181123 11:37:55 tornado_fetcher:638] fetcher starting...
[I 181123 11:37:55 processor:211] processor starting...
[I 181123 11:37:55 scheduler:647] scheduler starting...
[I 181123 11:37:55 scheduler:782] scheduler.xmlrpc listening on 127.0.0.1:23333
[I 181123 11:37:55 app:84] webui exiting...
[I 181123 11:37:55 scheduler:586] in 5m: new:0,success:0,retry:0,failed:0
Traceback (most recent call last):
File "/usr/bin/pyspider", line 9, in
- Deprecated option 'dir_browser.enable': use 'middleware_stack' instead.
@phongtnit @zekejames Have you addressed the issue?
I faced the similar problem as below:
Python 3.6.3 macOS Mojave 10.14.3
$ pyspider
[W 190305 09:17:33 run:413] phantomjs not found, continue running without it.
[I 190305 09:17:35 result_worker:49] result_worker starting...
[I 190305 09:17:36 processor:211] processor starting...
[I 190305 09:17:36 scheduler:647] scheduler starting...
[I 190305 09:17:36 tornado_fetcher:638] fetcher starting...
[I 190305 09:17:36 scheduler:782] scheduler.xmlrpc listening on 127.0.0.1:23333
[I 190305 09:17:36 scheduler:586] in 5m: new:0,success:0,retry:0,failed:0
[I 190305 09:17:36 app:84] webui exiting...
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.6/bin/pyspider", line 11, in <module>
load_entry_point('pyspider==0.3.10', 'console_scripts', 'pyspider')()
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pyspider/run.py", line 754, in main
cli()
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/click/core.py", line 1114, in invoke
return Command.invoke(self, ctx)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pyspider/run.py", line 165, in cli
ctx.invoke(all)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pyspider/run.py", line 497, in all
ctx.invoke(webui, **webui_config)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pyspider/run.py", line 384, in webui
app.run(host=host, port=port)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pyspider/webui/app.py", line 59, in run
from .webdav import dav_app
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pyspider/webui/webdav.py", line 216, in <module>
dav_app = WsgiDAVApp(config)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/wsgidav/wsgidav_app.py", line 135, in __init__
_check_config(config)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/wsgidav/wsgidav_app.py", line 119, in _check_config
raise ValueError("Invalid configuration:\n - " + "\n - ".join(errors))
ValueError: Invalid configuration:
- Deprecated option 'domaincontroller': use 'http_authenticator.domain_controller' instead.
WsgiDAV released a new version -- pre-release 3.x 'dir_browser.enable' DEPRECATED with 2.4.0 and 'domaincontroller' updated to 'http_authenticator.HTTPAuthenticator'
reference:https://wsgidav.readthedocs.io/en/latest/addons-ntdc.html
You can change pyspider/webui/webdav.py line 203:
config = DEFAULT_CONFIG.copy() config.update({ 'mount_path': '/dav', 'provider_mapping': { '/': ScriptProvider(app) }, #'domaincontroller': NeedAuthController(app), 'http_authenticator': { 'HTTPAuthenticator':NeedAuthController(app), }, 'verbose': 1 if app.debug else 0, 'dir_browser': {'davmount': False, #'enable': True, 'msmount': False, 'response_trailer': ''}, }) dav_app = WsgiDAVApp(config)
Last rebuild #python setup.py install
or You can try editing the version of wsgidav in ./setup.py
You are a fucking god.
Been troubleshooting this bad boy for about 3 hours now, this fixed it. https://i.imgur.com/Ohpmtre.png
except self change code, here reinstall old version also fix problem:
pipenv install wsgidav==2.4.1
except self change code, here reinstall old version also fix problem:
pipenv install wsgidav==2.4.1
thanks man!
except self change code, here reinstall old version also fix problem:
pipenv install wsgidav==2.4.1
this does the job!
except self change code, here reinstall old version also fix problem:
pipenv install wsgidav==2.4.1
Good! this works
WsgiDAV released a new version -- pre-release 3.x 'dir_browser.enable' DEPRECATED with 2.4.0 and 'domaincontroller' updated to 'http_authenticator.HTTPAuthenticator'
reference:https://wsgidav.readthedocs.io/en/latest/addons-ntdc.html
You can change pyspider/webui/webdav.py line 203:
config = DEFAULT_CONFIG.copy() config.update({ 'mount_path': '/dav', 'provider_mapping': { '/': ScriptProvider(app) }, #'domaincontroller': NeedAuthController(app), 'http_authenticator': { 'HTTPAuthenticator':NeedAuthController(app), }, 'verbose': 1 if app.debug else 0, 'dir_browser': {'davmount': False, #'enable': True, 'msmount': False, 'response_trailer': ''}, }) dav_app = WsgiDAVApp(config)
Last rebuild #python setup.py install
or You can try editing the version of wsgidav in ./setup.py
This works