Graceful handling of websocket closure

[barsuna]

Excellent project!

I've hit an issue when trying to test gpt-researcher with locally running llama3. When using firefox (125.0.2) on linux i see that on long research the browser closes the websocket (ping timeout?)

It shows up on the console like this:

🤔 Generating subtopics...

🤖 Calling gpt-4o...

📋Subtopics: subtopics=[...]
INFO:     connection closed

When this happens, next message that the agent tries to print is leading to agent 'freezing' - nothing else is happening after that point, i.e.

🔎 Starting the research task for '...'...

This does not happen with other browser (tried brave)

I cannot say i understand well what the relevant code does, but looking at server.py, i see that

...
    except WebSocketDisconnect:
        await manager.disconnect(websocket)
...

is getting invoked, just not sure that what is does is really handling the situation well - it seems that this exception is triggered with some delay in regards to browser closing the WS - by that time there is already a call to await stream_output which never returns.

for firefox i found a workaround, which is to increase 'network.websocket.timeout.ping.response' from 10 to 100 (need to go to 'about:config' url)