Malformed JSON object in request body crashes notebook_http personality

[ravenchilde]

If a malformed JSON object is sent in the body of an HTTP request, we've noticed that the kernel will return an error, then "reset" but any further requests to the TCP ports of the tornado server just don't ever respond. Browser just sits and spins, and curl just sits and waits.

When using notebook_http mode as a quickie API to some notebook data, users sending a wrongly formed JSON object is a definite possibility and it would be nice to catch that and just return an error and then free it up for another request.

Sent a curl request like: "curl -v -k -d ',{ "aaa_name":"AAAA999D", "aaa_aaa_address":"aaaa.aaaa.aaaa"}' -H "Content-Type: application/json" -X POST https://x.x.x.x:9697/stuff/morestuff/evenmorestuff/somethingsomething/aaa?flag=true"

Console:

[KernelGatewayApp] Jupyter Kernel Gateway at https://0.0.0.0:9697
ERROR:tornado.access:500 POST /stuff/morestuff/evenmorestuff/somethingsomething/aaa?flag=true (x.x.x.x) 29860.00ms
[KernelGatewayApp] KernelRestarter: restarting kernel (1/5), keep random ports
ERROR:tornado.general:Uncaught exception, closing connection.
Traceback (most recent call last):
  File "c:\python27\lib\site-packages\zmq\eventloop\zmqstream.py", line 414, in _run_callback
    callback(*args, **kwargs)
  File "c:\python27\lib\site-packages\tornado\stack_context.py", line 277, in null_wrapper
    return fn(*args, **kwargs)
  File "c:\python27\lib\site-packages\kernel_gateway\services\kernels\pool.py", line 145, in <lambda>
    return lambda msg_list: self._on_reply(kernel_id, session, msg_list)
  File "c:\python27\lib\site-packages\kernel_gateway\services\kernels\pool.py", line 124, in _on_reply
    self.on_recv_funcs[kernel_id](msg)
  File "c:\python27\lib\site-packages\kernel_gateway\notebook_http\handlers.py", line 108, in on_recv
    if msg['parent_header']['msg_id'] == parent_header:
KeyError: 'msg_id'
ERROR:tornado.general:Uncaught exception, closing connection.
Traceback (most recent call last):
  File "c:\python27\lib\site-packages\zmq\eventloop\zmqstream.py", line 440, in _handle_events
    self._handle_recv()
  File "c:\python27\lib\site-packages\zmq\eventloop\zmqstream.py", line 472, in _handle_recv
    self._run_callback(callback, msg)
  File "c:\python27\lib\site-packages\zmq\eventloop\zmqstream.py", line 414, in _run_callback
    callback(*args, **kwargs)
  File "c:\python27\lib\site-packages\tornado\stack_context.py", line 277, in null_wrapper
    return fn(*args, **kwargs)
  File "c:\python27\lib\site-packages\kernel_gateway\services\kernels\pool.py", line 145, in <lambda>
    return lambda msg_list: self._on_reply(kernel_id, session, msg_list)
  File "c:\python27\lib\site-packages\kernel_gateway\services\kernels\pool.py", line 124, in _on_reply
    self.on_recv_funcs[kernel_id](msg)
  File "c:\python27\lib\site-packages\kernel_gateway\notebook_http\handlers.py", line 108, in on_recv
    if msg['parent_header']['msg_id'] == parent_header:
KeyError: 'msg_id'
ERROR:tornado.application:Exception in callback None
Traceback (most recent call last):
  File "c:\python27\lib\site-packages\tornado\ioloop.py", line 888, in start
    handler_func(fd_obj, events)
  File "c:\python27\lib\site-packages\tornado\stack_context.py", line 277, in null_wrapper
   return fn(*args, **kwargs)
  File "c:\python27\lib\site-packages\zmq\eventloop\zmqstream.py", line 440, in _handle_events
    self._handle_recv()
  File "c:\python27\lib\site-packages\zmq\eventloop\zmqstream.py", line 472, in _handle_recv
    self._run_callback(callback, msg)
  File "c:\python27\lib\site-packages\zmq\eventloop\zmqstream.py", line 414, in _run_callback
    callback(*args, **kwargs)
  File "c:\python27\lib\site-packages\tornado\stack_context.py", line 277, in null_wrapper
    return fn(*args, **kwargs)
  File "c:\python27\lib\site-packages\kernel_gateway\services\kernels\pool.py", line 145, in <lambda>
    return lambda msg_list: self._on_reply(kernel_id, session, msg_list)
  File "c:\python27\lib\site-packages\kernel_gateway\services\kernels\pool.py", line 124, in _on_reply
    self.on_recv_funcs[kernel_id](msg)
  File "c:\python27\lib\site-packages\kernel_gateway\notebook_http\handlers.py", line 108, in on_recv
    if msg['parent_header']['msg_id'] == parent_header:
KeyError: 'msg_id'

[rolweber]

Small correction: it's the kernel gateway that returns the error. The message never gets through to the kernel.

Makes sense to add some error tolerance in this area. Any volunteers?

[dleen]

I made a notebook (endpoint.ipynb) with 3 cells:

import json
REQUEST = "{}"  # doesn't work without setting this somewhere

# GET /hello/world
print("hello world")

# POST /hello/world
req = json.loads(REQUEST)
print(req)

I started the kernel gateway using:

jupyter kernelgateway --KernelGatewayApp.api='kernel_gateway.notebook_http' \
                  --KernelGatewayApp.seed_uri=./endpoint.ipynb

and the output:

[KernelGatewayApp] Kernel started: a7008b5f-0b59-40d9-9318-ea33d4d89f8a
[KernelGatewayApp] Registering resource: /hello/world, methods: (['GET', 'POST'])
[KernelGatewayApp] Registering resource: /_api/spec/swagger.json, methods: (GET)
[KernelGatewayApp] The port 8888 is already in use, trying another port.
[KernelGatewayApp] Jupyter Kernel Gateway at http://127.0.0.1:8889

and finally ran:

curl -k -d ',{ "aaa_name":"AAAA999D", "aaa_aaa_address":"aaaa.aaaa.aaaa"}' -H "Content-Type: application/json" -X POST "http://127.0.0.1:8889/hello/world"

which resulted in the response:

{'body': ',{ "aaa_name":"AAAA999D", "aaa_aaa_address":"aaaa.aaaa.aaaa"}', 'args': {}, 'path': {}, 'headers': {'Host': '127.0.0.1:8889', 'User-Agent': 'curl/7.54.0', 'Accept': '*/*', 'Content-Type': 'application/json', 'Content-Length': '61'}}

and from the server logs:

INFO:tornado.access:200 POST /hello/world (127.0.0.1) 23.00ms

So it seems to be fine, unless there's something more going on in the notebook. I took a look at the last 10-20 commits in the package and nothing explicitly looks like it would fix this.

[ravenchilde]

@dleen - was happening on windows. Switched to Linux and the issue didn't replicate. Might have been something unique to the windows environment, or to Python on Windows.