Hippocampe
Hippocampe copied to clipboard
Published
20 hours ago •
TheHive-Project
TheHive-Project
Certain domains query cause Error 500
Request Type
Bug
Work Environment
| Question | Answer |
|---|---|
| OS version (server) | RedHat |
| OS version (client) | Seven, |
| Package Type | Docker |
Problem Description
Error 500 when asked for a domain doesok.top, baserpolaser.tk, letsgotohome.tk
Steps to Reproduce
- Query hipposcore for any of doesok.top, baserpolaser.tk, letsgotohome.tk
- Get error 500
curl -i -H "Content-Type: application/json" -X POST -d '{"ferasoplertyh.tk" : {"type" : "domain"} }' "https://domain.some/hippocampe/api/v1.0/hipposcore"
HTTP/1.0 500 INTERNAL SERVER ERROR
Content-Type: text/html
Content-Length: 291
Server: Werkzeug/0.14.1 Python/2.7.14
Date: Fri, 29 Jun 2018 07:44:28 GMT
Set-Cookie: XXXXXXX; path=/; HttpOnly; Secure
Connection: keep-alive
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>
Logs:
2018-06-29 08:00:43,423 :: services.more :: INFO :: {u'ferasoplertyh.tk': {u'type': u'domain'}}
2018-06-29 08:00:43,540 :: services.hipposcore :: INFO :: hipposcore.calcHipposcore launched
2018-06-29 08:00:43,545 :: services.hipposcore :: ERROR :: hipposcore.calcHipposcore failed, no idea where it came from...
Traceback (most recent call last):
File "/opt/Hippocampe/core/services/hipposcore.py", line 76, in calcHipposcore
P = P + (n3 * n2 * n1)
TypeError: unsupported operand type(s) for *: 'float' and 'NoneType'
2018-06-29 08:01:53,198 :: services.modules.common.ES :: INFO :: ES.checkES launched
Traceback (most recent call last):
resMsearch = searchIntel.littleMsearch(source.coreIntelligence, source.typeNameESIntel, parsedPage)
res = es.msearch(body = req)
File "/usr/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 76, in _wrapped
return func(*args, params=params, **kwargs)
File "/usr/lib/python2.7/site-packages/elasticsearch/client/__init__.py", line 1183, in msearch
headers={'content-type': 'application/x-ndjson'})
File "/usr/lib/python2.7/site-packages/elasticsearch/transport.py", line 314, in perform_request
status, headers_response, data = connection.perform_request(method, url, params, body, headers=headers, ignore=ignore, timeout=timeout)
File "/usr/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 175, in perform_request
raise ConnectionError('N/A', str(e), e)
ConnectionError: ConnectionError(('Connection aborted.', error(104, 'Connection reset by peer'))) caused by: ProtocolError(('Connection aborted.', error(104, 'Connection reset by peer')))
Yeah I had that issue too. This code base is a bit atrocious: Here is a super hacky solution:
add this after line 66, before hipposcoreDict = calcHipposcore(new_response)
logger.info("initiating fix")
try:
new_response = {}
for ip, data in response.iteritems():
temp_list = []
for item in data:
if item["idSource"] == '':
item["idSource"] = fixThisBrokenFunction(item["source"])
temp_list.append(item)
new_response[ip] = temp_list
except Exception as e:
logger.error('failed to fix their broken stuff', exc_info = True)
logger.error(response)
report = dict()
report['error'] = str(e)
and then right before if name == "main":
def fixThisBrokenBullshit(source):
es = getES()
data = {
'query' : {
'bool' : {
'must' : [
{'match' : {'source' : source}}
]
}
}
}
res = es.search(body=data)
for i in res['hits']['hits']:
if i["_source"]["idSource"] != "":
return i["_source"]["idSource"]
break
else:
continue
