Errors in httpd.py handling unexpected requests

[UnitedMarsupials-zz]

Some times, at the end of a run I get the following error on the Dispy-client's stderr:

Exception happened during processing of request from ('10.9.211.113', 54290)
Traceback (most recent call last):
  File "/prod/pfe/local/lib/python3.6/socketserver.py", line 320, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/prod/pfe/local/lib/python3.6/socketserver.py", line 351, in process_request
    self.finish_request(request, client_address)
  File "/prod/pfe/local/lib/python3.6/socketserver.py", line 364, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/prod/pfe/local/lib/python3.6/site-packages/dispy/httpd.py", line 379, in <lambda>
    self.__class__._HTTPRequestHandler(self, DocumentRoot, *args))
  File "/prod/pfe/local/lib/python3.6/site-packages/dispy/httpd.py", line 68, in __init__
    BaseHTTPRequestHandler.__init__(self, *args)
  File "/prod/pfe/local/lib/python3.6/socketserver.py", line 724, in __init__
    self.handle()
  File "/prod/pfe/local/lib/python3.6/http/server.py", line 418, in handle
    self.handle_one_request()
  File "/prod/pfe/local/lib/python3.6/http/server.py", line 406, in handle_one_request
    method()
  File "/prod/pfe/local/lib/python3.6/site-packages/dispy/httpd.py", line 170, in do_POST
    environ={'REQUEST_METHOD': 'POST'})
  File "/prod/pfe/local/lib/python3.6/cgi.py", line 566, in __init__
    self.read_single()
  File "/prod/pfe/local/lib/python3.6/cgi.py", line 757, in read_single
    self.read_binary()
  File "/prod/pfe/local/lib/python3.6/cgi.py", line 779, in read_binary
    self.file.write(data)
TypeError: write() argument must be str, not bytes

The web-interface is optional to my script, it is instantiated thus:

logger.info('Cluster created, attempting HTTP-server startup')
try:
	import dispy.httpd
	http = dispy.httpd.DispyHTTPServer(cluster)
	logger.info('HTTP-server created %s' % http._server)
except Exception as e:
	logger.error('Proceeding without HTTP-server: %s' % e)

At the end, when all submitted jobs have been processed, I do:

logger.info('Sleeping a bit to get any browser-monitoring a chance '
    'to update for one last time')
time.sleep(11)	# By default, browser updates every 10 seconds
cluster.print_status()
cluster.close(timeout = 0, terminate = True)

The exception does not affect the calculations -- indeed, it happens after everything is done and the client is exiting -- but it ruins my otherwise perfectly clean stderr...

I've seen this with earlier versions too, but the most recent incident was with python-3.6.8, dispy-4.10.5, pycos-4.8.11.

[UnitedMarsupials-zz]

I'm now convinced, this happens in responds to our company-wide security-checking (by Qualys). The normal usage of dispy's HTTP-server is fine, but attempts to exploit vulnerabilities known to exist in other HTTP server implementations causes this sort of exceptions.

Maybe, Dispy's httpd.py can be "hardened" to respond to invalid requests with a proper HTTP-error instead of exception?