Flask, SocketIOServer and debug mode

[abulte]

trafficstars

Hello,

I'm using gevent-socketio with Flask and serving it with socketio.server.SocketIOServer on my dev machine.

I need auto-reload and debug mode (Werkzeug error page when an error occurs) on this machine.

I can get autoreload to work, but not debug mode. When I enable debug mode like documented in Werkzeug (http://werkzeug.pocoo.org/docs/debug/), it simply stops serving websockets (no errors noticeable).

Below is one of my multiple tries at making it work (I also tried to serve a DebuggedApplication and a standard app through DispatcherMiddleware but that did not work either).

Any idea ? Thanks.

from buildmyios import app
from gevent import monkey
from socketio.server import SocketIOServer
import werkzeug.serving
from werkzeug.debug import DebuggedApplication

# necessary for autoreload (at least)
monkey.patch_all()

PORT = 5000

@werkzeug.serving.run_with_reloader
def runServer():
    print 'Listening on %s...' % PORT
    ws = SocketIOServer(('0.0.0.0', PORT), DebuggedApplication(app), resource="socket.io", policy_server=False)
    ws.serve_forever()

runServer()

[TronPaul]

I think there's something with the way gevent works that does not play nice with werkzeug's debugging (something to do with the weird stack traces when you have multiple workers). You might try and set workers to 1 when you're debugging in your development environment.

[grizwako]

@TronPaul , I did not have problems with werkezug debugger and gevent. I did some basic tests and all was fine (launched couple of greenlets all bound to different url, some were sleeping ,some were doing random calculations and one had tiny sleep and NameError done on purpose). I spent some time just now trying to figure what am I doing wrong with socket.io/ gevent-socketio and slowly started to think that i am going insane.. Eventually I figured that werkzeug debugger is only difference between sample that i was doing and chat example from this repo.

[sontek]

I'm not too familiar with how werkzeug works but if you can provide me a sample application that shows the problem I can dig into it.

[abourget]

Please read invitation to Wednesday September 18th's sprint: https://groups.google.com/forum/#!topic/gevent-socketio/2OIRKA8M2uE

[Lily418]

I was trying to do the same thing and got it to work: https://gist.github.com/anonymous/6687587

[abulte]

Nice gist, thanks!

[aldanor]

@abulte @JoelHoskin doesn't seem to work for me, neither with WSGIServer nor with SocketIOServer, still breaks websockets.. is there any way?

I mean, this minimal example does work as it is, but if you actually to do any work with sockets it will fail. Correct me if I'm wrong?

[Lily418]

@Aldanor I've expanded the Gist into a repo https://github.com/JoelHoskin/SocketIO-Flask-Debug This application is working when I test it on my machine.

Hopefully you can reproduce this by: git clone https://github.com/JoelHoskin/SocketIO-Flask-Debug.git pip install Flask gevent-socketio Open http://127.0.0.1:8080 in browser Then in a new tab open http://127.0.0.1:8080/doAFoo which should bring up an alert on http://127.0.0.1:8080 showing Socket.IO in action. Then http://127.0.0.1:8080/brokenPage brings up a "ValueError: View function did not return a response" with full stack-trace showing Flask debugging is working.

If you can't reproduce this then this issue is only affecting certain set-ups. If you can reproduce this then perhaps this issue is fixed?

[aldanor]

@JoelHoskin Thanks for posting a repo!

Well, it works halfway, I get the debugger page if I browser to /brokenPage, however if I browse to /doAFoo, foo() gets called but there's no alert box on the client side...

[aldanor]

@abulte @JoelHoskin @sontek @abourget I've created another example based on Joel's, which is a tiny bit more transparent and logs everything to Python/JS consoles:

https://github.com/aldanor/SocketIO-Flask-Debug (see the README.md for description)

Would anyone be able to confirm that this example works / doesn't work?

By looking at the console messages, I guess the server never executes the handler for route "/socket.io/<path:path>", hence socketio_manage never gets called, wonder why.

Update: on some further inspection, the generator returned by DebuggedApplication.debug_application (https://github.com/mitsuhiko/werkzeug/blob/master/werkzeug/debug/init.py#L90), which is in its turn is called from DebuggedApplication.__call__, gets never called, so its results are never retrieved.

In fact it does generate a valid response in the first place for socketio requests, which can be verified by inserting some test code like response(environ, start_response).next() in DebuggedApplication.__call__, then the websocket handlers would get called and the app that I posted above would kinda work. Alternatively, if debug_application is patched so it instantly collects results from the generator and returns them, it also kinda works (on the gevent-socketio side), but then the debug is broken because all the other werkzeug methods expect a generator.

Which leads to the question, why doesn't it get called? May that be because gevent-socketio doesn't care about what WSGI app __call__ method returns, so it never bothers to iterate over the generator? (unlike the generic handle_one_response which calls process_request that collects the results in a for loop, hence the generator actually gets iterated over)

Update 2: for the initial socket connection request, it is indeed possible and quite easy to hack handler.handle_one_response by binding the completion of socket.wsgi_app_greenlet to a function which iterates over its value if its a generator -- this makes sockets work, I can post an example if anyone's interested. Werkzeug debugger would also work for normal flask requests, however any exception raised within gevent-socketio namespaces would be unhandled, and to be honest I have no clue how to do that in a simple way. Even if we extract the traceback, we need to somehow push this to the client.

[aldanor]

Ok, holy crap. Looks like I finally made it work without messing with gevent-socketio internals... it requires several steps:

• subclass werkzeug.DebuggedApplication and make __call__ method iterate over the result if we are dealing with web sockets and the returned value is a generator
• set BaseNamespace.exception_handler_decorator to catch exceptions and store them in some array, then emit "exception" message to client
• in the client, reload the page upon "exception" socket message (or do a custom GET request, or forward to specific url)
• in flask, raise the exception instead of rendering the normal html if the exception list is non-empty; werkzeug will pick it up from there and do its thing with correct traceback

I can push an example tomorrow if anyone's interested.

There's still some design questions though... like where should these exceptions be stored so they can be seen from both the namespace and the app? The namespace doesn't have a reference to the flask app, does it? I'm just wondering what would be the least ugly implementation (up to pickling exceptions and sending them via base64 to the client so they can get sent back via post requests, whatever).

[limodou]

I also encountered this problem, and I would like to see your testing code. Thanks a lot. And I add some testing code at the begining of DebugApplication.__call__, but the code is not executed at all. According to your description above, gevent-socketio could invoke DebugApplication.__call__, and the case seems not suit for me.

[limodou]

I found the answer http://stackoverflow.com/questions/18319345/gevent-socketio-not-using-my-app-route-endpoint-for-socketio

[aldanor]

@limodou That's not quite a solution, more like skipping the debugger upon connection; try throwing an exception from within your socket.io namespace and see what happens.

[limodou]

Yes, if there is exception occured, the Debug Page will not available like other urls. But for now I can use DebugApplication at least. Waiting for your solution.

[aldanor]

Ok guys, enjoy, after many hours of painful debugging I kinda seemed to solve it:

https://github.com/aldanor/SocketIO-Flask-Debug

Sockets work as intended and the debugger handles both flask exceptions and namespace exceptions (before they get consumed by greenlets). Werkzeug's auto-reloader now also works. The only real usage difference is that you have to force reload the page manually on the client side if there was a namespace exception; however, that's just one line of code.

Wonder if it's worth wrapping it as a pypi package or something.

[limodou]

I've seen the code, great.

But it's a little tight with Flask I think. e.g. hasattr(self.application, 'before_request')

Other problem:

            if hasattr(self.app, 'before_request'):
                self.app.before_request(self.route_debugger)

I changed to self.application . I don't know if my DebugApplication code of werkzeug is not like you.

I don't understand route_debugger very much. Why not just raise in protected_method, will this break the greenlet? And will the debug page be displayed at the next visit?

[aldanor]

@limodou Indeed, before_request is a Flask feature, I'm no expert in other frameworks so I've implemented it in a Flask way. However, we could use werkzeug.routing module to intercept a request before it gets passed to the WSGI app, this way it would be framework-agnostic. Or actually, wait, we can even put it into the __call__ method itself. I'll look into it later today.

route_debugger might be a misleading name, all it does is checks is if there's a saved exception and reraises it with the original traceback. It's injected into before_request, so it will get called when you reload the page on the JS side.

You can't raise the exception in protected_method since it would be raised in a greenlet thread and hence consumed by gevent. The whole point is getting it out of there and reraising it in the application thread.

[limodou]

I saw you said "reload", so does it mean only when user refresh the page it'll see the exception, not like other debug pages which will be displayed at the same visit. But when I run the test code, I did see the debug page is displayed after I click "Raise a namespace exception" button. Whether it is because the browser will reconnect automatically?

[aldanor]

@limodou There is no "automatically", please see the "client" part of README.md in the repo, and also the JS code in the index.html file. We actually need to force the client to send us a request so we can raise an exception in its context. That's why the "exception" message is emitted through the socket, and then the client instantly reloads the page -- this request is intercepted by the debugger and you get a debug page. So it's not quite a one-step process.

[limodou]

Oh, I see. So the debug page should be cooperate with client. And I also want to ask a question: if there are some concurrency requests, so is it possible that the other user can get the debug page? Should your method consider using session to enable only the original user can get the exception?

[aldanor]

Well, whoever reloads the page first will see the debug page, and the "exception" message is emitted only to the user whose socket.io namespace triggered the exception, so in practice it's doubtful that another user will see it? But possible in theory. Idk how this should be handled though... use the BaseNamespace's environ/session and save them together with the exception -- will that work? I'm not sure. We can generate a random uuid and attach it to the "exception" socket message, then the client would have to browse to a custom /uuid link to get the exception. Or would that be too ugly?

[limodou]

Yes,it seems ugly and complication. I also have good idea.

[limodou]

I've thougth concurrency problem again, maybe it's not seriously so much. We will use debug mode commonly in development environment.