Scan-T
Scan-T copied to clipboard
nanshihui
关于安装方面的问题汇总
目前遇到的问题都是 ubuntu:latest 环境。
相关程序的安装：
apt-get update
apt-get install -y nmap
apt-get install -y libjson-c-dev libjson-c2 libjson0 libjson0-dev
apt-get install -y redis-server zmap libffi-dev libssl-dev python-pip libmysqlclient-dev
apt-get install -y wget unzip
获取代码
wget https://github.com/nanshihui/toolforspider/archive/master.zip
unzip master.zip
cd toolforspider-master/
安装相关的package
pip install MySQL-python
pip install BeautifulSoup==3.2.1
pip install beautifulsoup4==4.4.1
pip install Django==1.9
pip install python-nmap==0.5.0.post1
pip install DBUtils==1.1
pip install paramiko==1.16.0
pip install ruamel.ordereddict==0.4.9
pip install scapy==2.3.2
pip install scapy-http==1.7
pip install objgraph==2.0.1
pip install pycrypto==2.6.1
pip install dozer
pip install faulthandler
pip install apscheduler
配置
数据库配置位于 spidermanage/spidertool/config.py
如果本机没有装mysql，可以按照如下方式进行安装。
apt-get install mysql-server
启动mysql服务
/etc/init.d/mysql start
建立数据库
mysql -uroot -p -e 'create database datap'
启动
先迁移下数据库
python spidermanage/manage.py migrate
mysql -uroot -p datap -e 'source /toolforspider-master/spidermanage/sqldata/Dump20160331.sql'
注意：因为环境问题，我去掉了/*!50100 WITH PARSER mysqlcft */ 这段。
mysqlcft 比较老旧，怕以后有兼容性问题，所以不打算继续折腾。
添加网站账户
mysql -uroot -p datap -e "insert INTO user_table set username = 'admin',password = 'admin'"
正式启动
python spidermanage/manage.py runserver 0.0.0.0:80
登录后你会发现有些页面点击进去会发生错误，那是因为作者有些功能还没有完成。
在 /toolforspider-master/spidermanage/common_static/nmaptool/css/bootstrap/ 下 新建 lib 文件夹 并且下载 http://cine9deabril.com.br/fn/jquery/jquery-ui-1.10.2.custom/css/custom-theme/jquery-ui-1.10.2.custom.css https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.6.3/css/font-awesome.css 这两个文件到文件夹里
使用 python spidermanage/manage.py runserver localhost:80 运行后出现如下错误：
Performing system checks...
Unhandled exception in thread started by <function wrapper at 0x7f98ed8368c0>
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/django/utils/autoreload.py", line 226, in wrapper
fn(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/runserver.py", line 116, in inner_run
self.check(display_num_errors=True)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 426, in check
include_deployment_checks=include_deployment_checks,
File "/usr/local/lib/python2.7/dist-packages/django/core/checks/registry.py", line 75, in run_checks
new_errors = check(app_configs=app_configs)
File "/usr/local/lib/python2.7/dist-packages/django/core/checks/urls.py", line 10, in check_url_config
return check_resolver(resolver)
File "/usr/local/lib/python2.7/dist-packages/django/core/checks/urls.py", line 19, in check_resolver
for pattern in resolver.url_patterns:
File "/usr/local/lib/python2.7/dist-packages/django/utils/functional.py", line 33, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 417, in url_patterns
patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
File "/usr/local/lib/python2.7/dist-packages/django/utils/functional.py", line 33, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 410, in urlconf_module
return import_module(self.urlconf_name)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/toolforspider-master/spidermanage/spidermanage/urls.py", line 23, in <module>
url(r'^nmaptool/', include('nmaptoolbackground.urls',namespace='nmaptool')),
File "/usr/local/lib/python2.7/dist-packages/django/conf/urls/__init__.py", line 52, in include
urlconf_module = import_module(urlconf_module)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/toolforspider-master/spidermanage/nmaptoolbackground/urls.py", line 5, in <module>
from . import nmaproute as route
File "/toolforspider-master/spidermanage/nmaptoolbackground/nmaproute.py", line 8, in <module>
from control import usercontrol,jobcontrol,ipcontrol,portcontrol,taskcontrol
File "/toolforspider-master/spidermanage/nmaptoolbackground/control/taskcontrol.py", line 3, in <module>
from spidertool import sniffertask,scapytask
File "/toolforspider-master/spidermanage/spidertool/sniffertask.py", line 8, in <module>
import sniffertool
File "/toolforspider-master/spidermanage/spidertool/sniffertool.py", line 20, in <module>
import portscantask
File "/toolforspider-master/spidermanage/spidertool/portscantask.py", line 7, in <module>
import portscantool
File "/toolforspider-master/spidermanage/spidertool/portscantool.py", line 11, in <module>
from template_identify import port_identify
File "/toolforspider-master/spidermanage/spidertool/template_identify/port_identify.py", line 5, in <module>
from poc_file import pocsearchtask
File "/toolforspider-master/spidermanage/spidertool/template_identify/poc_file/__init__.py", line 6, in <module>
from lib.logger import initLog
ImportError: No module named lib.logger
临时解决办法
mkdir spidermanage/lib/
touch spidermanage/lib/__init__.py
在 spidermanage/lib/ 中新建 logger.py 内容如下
def initLog(a,b,c):
pass
继续尝试运行，依然报错：
Performing system checks...
WARNING: Failed to execute tcpdump. Check it is installed and in the PATH
WARNING: No route found for IPv6 destination :: (no default route?)
Unhandled exception in thread started by <function wrapper at 0x7f0df1bd38c0>
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/django/utils/autoreload.py", line 226, in wrapper
fn(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/runserver.py", line 116, in inner_run
self.check(display_num_errors=True)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 426, in check
include_deployment_checks=include_deployment_checks,
File "/usr/local/lib/python2.7/dist-packages/django/core/checks/registry.py", line 75, in run_checks
new_errors = check(app_configs=app_configs)
File "/usr/local/lib/python2.7/dist-packages/django/core/checks/urls.py", line 10, in check_url_config
return check_resolver(resolver)
File "/usr/local/lib/python2.7/dist-packages/django/core/checks/urls.py", line 19, in check_resolver
for pattern in resolver.url_patterns:
File "/usr/local/lib/python2.7/dist-packages/django/utils/functional.py", line 33, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 417, in url_patterns
patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
File "/usr/local/lib/python2.7/dist-packages/django/utils/functional.py", line 33, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 410, in urlconf_module
return import_module(self.urlconf_name)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/toolforspider-master/spidermanage/spidermanage/urls.py", line 23, in <module>
url(r'^nmaptool/', include('nmaptoolbackground.urls',namespace='nmaptool')),
File "/usr/local/lib/python2.7/dist-packages/django/conf/urls/__init__.py", line 52, in include
urlconf_module = import_module(urlconf_module)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/toolforspider-master/spidermanage/nmaptoolbackground/urls.py", line 5, in <module>
from . import nmaproute as route
File "/toolforspider-master/spidermanage/nmaptoolbackground/nmaproute.py", line 8, in <module>
from control import usercontrol,jobcontrol,ipcontrol,portcontrol,taskcontrol
File "/toolforspider-master/spidermanage/nmaptoolbackground/control/taskcontrol.py", line 4, in <module>
import schedule
File "/toolforspider-master/spidermanage/nmaptoolbackground/control/schedule.py", line 3, in <module>
from apscheduler.schedulers.background import BackgroundScheduler
ImportError: No module named apscheduler.schedulers.background
临时解决办法
pip install apscheduler
继续运行，报错
May 27, 2016 - 13:03:45
Django version 1.9, using settings 'spidermanage.settings'
Starting development server at http://localhost:80/
Quit the server with CONTROL-C.
Unhandled exception in thread started by <function wrapper at 0x7fb197d5e668>
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/django/utils/autoreload.py", line 226, in wrapper
fn(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/runserver.py", line 135, in inner_run
handler = self.get_handler(*args, **options)
File "/usr/local/lib/python2.7/dist-packages/django/contrib/staticfiles/management/commands/runserver.py", line 23, in get_handler
handler = super(Command, self).get_handler(*args, **options)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/runserver.py", line 60, in get_handler
return get_internal_wsgi_application()
File "/usr/local/lib/python2.7/dist-packages/django/core/servers/basehttp.py", line 57, in get_internal_wsgi_application
sys.exc_info()[2])
File "/usr/local/lib/python2.7/dist-packages/django/core/servers/basehttp.py", line 47, in get_internal_wsgi_application
return import_string(app_path)
File "/usr/local/lib/python2.7/dist-packages/django/utils/module_loading.py", line 20, in import_string
module = import_module(module_path)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/toolforspider-master/spidermanage/spidermanage/wsgi.py", line 17, in <module>
from dozer import Dozer,Logview
django.core.exceptions.ImproperlyConfigured: WSGI application 'spidermanage.wsgi.application' could not be loaded; Error importing module: 'No module named dozer'
解决办法 pip install dozer pip install faulthandler
lucy@lucy:~/123/toolforspider$ python spidermanage/manage.py migrate
Traceback (most recent call last):
File "spidermanage/manage.py", line 10, in
lucy@lucy:~/toolforspider$ python spidermanage/manage.py migrate
WARNING: No route found for IPv6 destination :: (no default route?)
Traceback (most recent call last):
File "spidermanage/manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 350, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 342, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 348, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 398, in execute
self.check()
File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 426, in check
include_deployment_checks=include_deployment_checks,
File "/usr/local/lib/python2.7/dist-packages/django/core/checks/registry.py", line 75, in run_checks
new_errors = check(app_configs=app_configs)
File "/usr/local/lib/python2.7/dist-packages/django/core/checks/urls.py", line 10, in check_url_config
return check_resolver(resolver)
File "/usr/local/lib/python2.7/dist-packages/django/core/checks/urls.py", line 19, in check_resolver
for pattern in resolver.url_patterns:
File "/usr/local/lib/python2.7/dist-packages/django/utils/functional.py", line 33, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 417, in url_patterns
patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
File "/usr/local/lib/python2.7/dist-packages/django/utils/functional.py", line 33, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 410, in urlconf_module
return import_module(self.urlconf_name)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/home/lucy/toolforspider/spidermanage/spidermanage/urls.py", line 23, in <module>
url(r'^nmaptool/', include('nmaptoolbackground.urls',namespace='nmaptool')),
File "/usr/local/lib/python2.7/dist-packages/django/conf/urls/__init__.py", line 52, in include
urlconf_module = import_module(urlconf_module)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/home/lucy/toolforspider/spidermanage/nmaptoolbackground/urls.py", line 5, in <module>
from . import nmaproute as route
File "/home/lucy/toolforspider/spidermanage/nmaptoolbackground/nmaproute.py", line 8, in <module>
from control import usercontrol,jobcontrol,ipcontrol,portcontrol,taskcontrol
File "/home/lucy/toolforspider/spidermanage/nmaptoolbackground/control/taskcontrol.py", line 4, in <module>
import schedule
File "/home/lucy/toolforspider/spidermanage/nmaptoolbackground/control/schedule.py", line 3, in <module>
from apscheduler.schedulers.background import BackgroundScheduler
File "/usr/local/lib/python2.7/dist-packages/apscheduler/__init__.py", line 3, in <module>
version_info = tuple(int(x) if x.isdigit() else x for x in parsed_version.public.split('.'))
AttributeError: 'tuple' object has no attribute 'public'
lucy@lucy:~/toolforspider$
Before you run it ,you have to make sure that all the environment you have already installed , then you can follow my wiki introduction step by step.
1.toolforspider/spidermanage/spidermanage/settings.py~要在这里添加数据库信息，这一步是不是没写啊 2.登录的时候返回django的debug页面说没有/toolforspider/spidermanage/logs/sqltool.log这个文件，新建一个就好了
WARNING: No route found for IPv6 destination :: (no default route?)
Traceback (most recent call last):
File "spidermanage/manage.py", line 10, in
静态文件全部404,报这个WARNINGS: ?: (1_8.W001) The standalone TEMPLATE_* settings were deprecated in Django 1.8 and the TEMPLATES dictionary takes precedence. You must put the values of the following settings into your default TEMPLATES dict: TEMPLATE_DIRS. 请问怎么解决啊？
如果只是出现这句话 (1_8.W001) The standalone TEMPLATE_* settings were deprecated in Django 1.8 and the TEMPLATES dictionary takes precedence. You must put the values of the following settings into your default TEMPLATES dict: TEMPLATE_DIRS. 那是没有什么问题的。
如果静态文件全是404，请按wiki的安装方式来。
静态文件全部404我的解决方法：
1.在Scan-T-master\spidermanage\spidermanage\urls.py里面添加一条路由：
url(r'^static/(?P
然后成功~
我和@xiaoxiaoleo 一样python spidermanage/manage.py runserver 0.0.0.0:80后 也出现了 WARNINGS: ?: (1_8.W001) The standalone TEMPLATE_* settings were deprecated in Django 1.8 and the TEMPLATES dictionary takes precedence. You must put the values of the following settings into your default TEMPLATES dict: TEMPLATE_DIRS.
System check identified 1 issue (0 silenced). June 17, 2016 - 12:18:17 Django version 1.9, using settings 'spidermanage.settings' Starting development server at http://0.0.0.0:80/ Quit the server with CONTROL-C. WARNING:py.warnings:/usr/local/lib/python2.7/dist-packages/dozer/leak.py:97: UserWarning: PIL is not installed, cannot show charts in Dozer warnings.warn('PIL is not installed, cannot show charts in Dozer')
init schedule
取消后出现 @ip-172-31-28-150:~/Scan-T-master# Exception in thread APScheduler (most likely raised during interpreter shutdown): Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner File "/usr/lib/python2.7/threading.py", line 763, in run File "/usr/local/lib/python2.7/dist-packages/apscheduler/schedulers/blocking.py", line 28, in _main_loop File "/usr/lib/python2.7/threading.py", line 620, in wait File "/usr/lib/python2.7/threading.py", line 354, in wait <type 'exceptions.TypeError'>: 'NoneType' object is not callable
用了@xiaoxiaoleo的解决方法，python spidermanage/manage.py runserver 0.0.0.0:80后出现
root@ip-172-31-28-150:~/Scan-T-master# python spidermanage/manage.py runserver 0.0.0.0:80 --insecure Performing system checks...
WARNING: No route found for IPv6 destination :: (no default route?) WARNING:py.warnings:/root/Scan-T-master/spidermanage/spidermanage/urls.py:27: RemovedInDjango110Warning: Support for string view arguments to url() is deprecated and will be removed in Django 1.10 (got django.views.static.serve). Pass the callable instead. url(r'^static/(?P.*)$', 'django.views.static.serve',{'document_root': "common_static"}),
Unhandled exception in thread started by <function wrapper at 0x7f73a894eb18> Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/django/utils/autoreload.py", line 226, in wrapper fn(_args, _kwargs) File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/runserver.py", line 116, in inner_run self.check(display_num_errors=True) File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 426, in check include_deployment_checks=include_deployment_checks, File "/usr/local/lib/python2.7/dist-packages/django/core/checks/registry.py", line 75, in run_checks new_errors = check(app_configs=app_configs) File "/usr/local/lib/python2.7/dist-packages/django/core/checks/urls.py", line 10, in check_url_config return check_resolver(resolver) File "/usr/local/lib/python2.7/dist-packages/django/core/checks/urls.py", line 27, in check_resolver warnings.extend(check_pattern_startswith_slash(pattern)) File "/usr/local/lib/python2.7/dist-packages/django/core/checks/urls.py", line 63, in check_pattern_startswith_slash regex_pattern = pattern.regex.pattern File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 203, in regex (regex, six.text_type(e))) django.core.exceptions.ImproperlyConfigured: "^static/(?P.)$" is not a valid regular expression: unknown specifier: ?P.
还有系统可能需要安装gevent For Python 具体方法 下载gevent源码，并解压 wget http://pypi.python.org/packages/source/g/gevent/gevent-0.13.1.tar.gz#md5=5c1b03d9ce39fee4cfe5ea8befb1d4c4 tar -zxvf gevent-0.13.1.tar.gz 安装编译 cd gevent-0.13.1 python fetch_libevent.py python setup.py build sudo python setup.py install
@iwarsong 我的解决方法不是完美的处理静态文件404的方案，http://127.0.0.1:8000/admin目录由于调用django自带的静态文件导致静态文件仍然是404。我的django版本是1.9。
@iwarsong 执行时候加一个--insecure试试。 python manage.py runserver --insecure
@iwarsong @xiaoxiaoleo django 默认在set debug 标签为false 的时候，静态路由 是交给 负载均衡器 来做的，你可以加一个nginx对静态文件路由，或者按着我的wiki 上的说明
还有是否还得安装PIL~ 下载PIL的Source Kit（因为这个包支持全部平台） Imaging--1.1.7.tar.gz URL: http://www.pythonware.com/products/pil/index.htm
2、解压缩包 tar -zxvf Imaging-1.1.7.tar.gz
3、进入到解压后的目录 cd Imaging-1.1.7 4、Build pakage: python setup.py build_ext -i
5、测试; python selftest.py
6、安装 python setup.py install
mysql一共三处地址需要配置: Scan-T-master\Scan-T-master\spidermanage\spidertool\config.py Scan-T-master\Scan-T-master\spidermanage\spidermanage\settings.py Scan-T-master\Scan-T-master\spidermanage\spidertool\scapytool.py
@nanshihui
@iwarsong @xiaoxiaoleo django 默认在set debug 标签为false 的时候，静态路由 是交给 负载均衡器 来做的，你可以加一个nginx对静态文件路由，或者按着我的wiki 上的说明
这方面我不是很懂~能否给个过程~thx~
spidermanage/manage.py runserver 0.0.0.0:80 insecure Performing system checks...
Unhandled exception in thread started by <function wrapper at 0x7fa1475da758>
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/django/utils/autoreload.py", line 226, in wrapper
fn(_args, *_kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/runserver.py", line 116, in inner_run
self.check(display_num_errors=True)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 426, in check
include_deployment_checks=include_deployment_checks,
File "/usr/local/lib/python2.7/dist-packages/django/core/checks/registry.py", line 75, in run_checks
new_errors = check(app_configs=app_configs)
File "/usr/local/lib/python2.7/dist-packages/django/core/checks/urls.py", line 10, in check_url_config
return check_resolver(resolver)
File "/usr/local/lib/python2.7/dist-packages/django/core/checks/urls.py", line 19, in check_resolver
for pattern in resolver.url_patterns:
File "/usr/local/lib/python2.7/dist-packages/django/utils/functional.py", line 33, in get
res = instance.dict[self.name] = self.func(instance)
File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 417, in url_patterns
patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
File "/usr/local/lib/python2.7/dist-packages/django/utils/functional.py", line 33, in get
res = instance.dict[self.name] = self.func(instance)
File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 410, in urlconf_module
return import_module(self.urlconf_name)
File "/usr/lib/python2.7/importlib/init.py", line 37, in import_module
import(name)
File "/root/Scan-T-master/spidermanage/spidermanage/urls.py", line 23, in
求解
把portscantool.py 第11行的from detection import port_identify 移到26行后面, 如果觉得麻烦的话,你可以不用修改 ,我晚些时候更新.