uWebSockets.js icon indicating copy to clipboard operation
uWebSockets.js copied to clipboard

What can cause this error "Cork buffer must not be held across event loop iterations!"?

Open amunhoz opened this issue 4 years ago • 25 comments

I'm using uWebsockets.js to communicate between servers, and i got this error:

"Cork buffer must not be held across event loop iterations!"

What exactly can cause this error in my js code?

amunhoz avatar Sep 14 '21 23:09 amunhoz

Do you have some code that can trigger this issue?

ghost avatar Sep 14 '21 23:09 ghost

No, i'm using it for a tunnel in a socks proxy with a lot of operations. I could not replicate it after the first one.

amunhoz avatar Sep 15 '21 00:09 amunhoz

You compiled your own version from source? Or are you using C++?

ghost avatar Sep 15 '21 00:09 ghost

Nope that doesn't fix anything. Can you show anything that reproduces it?

ghost avatar Sep 15 '21 01:09 ghost

Compiled from the source in ubuntu 18.04 from cloned repo. But i cant get any code to reproduce it.

amunhoz avatar Sep 15 '21 12:09 amunhoz

Can you give any more info - what parts are you using? Are you using uWS's pub/sub? Are you doing anything strange like forking or such? Are you making use of cork functions? How often does it happen? Etc.

ghost avatar Sep 16 '21 13:09 ghost

More info:

  • Compiled the code from the latest github repo
  • Only using websocket (sending, receiving data and geting ip info)
  • VMware VM + Ubuntu 18.04
  • I've tested with a remote connection and tried to load some work, didn't got any error
  • The error occured at local tests loading simple pages with the proxy (really low cpu) Sorry, really dont know what caused it. I'm closing since we cant find it.

amunhoz avatar Sep 16 '21 17:09 amunhoz

Only using websocket (sending, receiving data and geting ip info)

Can you say Yes/No if you are a) using publish, subscribe, unsubscribe functions? b) cork functions? If you don't know what those are, and you only use send functions - that's a No.

ghost avatar Sep 16 '21 20:09 ghost

Not used publish, subscribe, unsubscribe and cork functions..

This is the code (removed some parts):

const { nanoid } = require('nanoid')
const uws =  require('uws-compiled-by-me')

module.exports = class proxyTransport extends base {
    constructor(options){
        super()        
        this.options = options || {}
        this.options.timeout = this.options.timeout || 10000
        this.options.ping = this.options.ping || 10000        
        this.clients = {}
        this.online =  false
    }
    async listen(address){
        this.type = "server"
        let parts = address.split(":");
        let port = parts[parts.length-1]             
        this.options.port = parseInt(port)
        this.server = uws.App({})     

        var self = this;
        this.server.ws('/*', {
            maxPayloadLength: 10 * 1024 * 1024,
            idleTimeout: 0,
            open: (socket) => {
                let id = nanoid(6);
                socket.id = id;
                socket.readyState = 1
                socket._send = socket.send
                socket.send = function(data){
                    socket._send(data, true)
                }
                self.clients[socket.id] = socket                
            },
            upgrade: async (res, req, context) => {           
                let ip = self._getIp(res.getRemoteAddress(), res.getRemoteAddressAsText())                            
                res.upgrade({ip: ip},
                      /* Spell these correctly */
                      req.getHeader('sec-websocket-key'),
                      req.getHeader('sec-websocket-protocol'),
                      req.getHeader('sec-websocket-extensions'),
                      context);
            },
            message: (socket, data, isBinary) => {              
                self.emit("data", Buffer.from(data), socket.id)                
            },            
            close: (socket, code, message) => {
                socket.readyState = 0
                delete this.clients[socket.id]               
            }
        }).any('/*', (res, req) => {
                res.writeStatus('404 NOT_FOUND').end('not here!');
        })
        
        self.server.listen(this.options.port , (token) => {
            if (token) {
                self.online = true
                return true
            }
            else return false
        }) 
    }   

    _getIp(ipBin, ipText){            
        let ipBuf = Buffer.from(ipBin)
        let isIpv4 = true        
        for (let i in ipv4Array) {    
            if (typeof ipBuf[i] == "undefined" || ipBuf[i] != ipv4Array[i]) {
                isIpv4 = false;
                break;
            }                
        }                        
        if (isIpv4) {
            //ipv4     
            ipBuf = ipBuf.slice(12)       
            return `${ipBuf[0]}.${ipBuf[1]}.${ipBuf[2]}.${ipBuf[3]}`
        } else {
            //ipv6
            return Buffer.from(ipText).toString()            
        }
    }

    socketClose(id){
        if (!this.clients[id]) return false
        this.clients[id].close()
        delete this.clients[id]
    }

    send(data, socket) {
        if (!this.online) throw new Error("Not connected")        
        if (!this.clients[socket]) return        

        this.clients[socket].send(data)
        return true
    }
    close() {
        this.closed = true
        this.server.close()      
    }

}
const ipv4Array = [0,0,0,0,0,0,0,0,0,0,255,255]

amunhoz avatar Sep 16 '21 20:09 amunhoz

I was able to repro it by calling res.upgrade inside of res.cork while inside a timer or anything that isn't a message handler or similar.

You are calling res.upgrade in an async function -> do you await on anything in that self._getIp? I'm pretty sure the issue is around res.upgrade that happens "async", outside of uWS's handlers.

ghost avatar Sep 16 '21 20:09 ghost

Yes, i was calling a async function (i removed from the code above). So, if i call res.upgrade from a callback or something else, will i get the same error? Is this something which can be fixed or do i have to change my code?

the original code:

upgrade: async (res, req, context) => {           
                let ip = self._getIp(res.getRemoteAddress(), res.getRemoteAddressAsText())
                if (await self._checkFilter(ip) == true) {  // this is a way to interact with a firewall code to block ips
                    return res.close()
                }                
                res.upgrade({ip: ip},
                      /* Spell these correctly */
                      req.getHeader('sec-websocket-key'),
                      req.getHeader('sec-websocket-protocol'),
                      req.getHeader('sec-websocket-extensions'),
                      context);
            },

Can i solve with this?


upgrade: (res, req, context) => {           
                let ip = self._getIp(res.getRemoteAddress(), res.getRemoteAddressAsText())
                self._checkFilter(ip).then((BlockIp)=>{
                    if (BlockIp) return res.close()
                    res.upgrade({ip: ip},
                        /* Spell these correctly */
                        req.getHeader('sec-websocket-key'),
                        req.getHeader('sec-websocket-protocol'),
                        req.getHeader('sec-websocket-extensions'),
                        context);
                })                                
            },

amunhoz avatar Sep 16 '21 21:09 amunhoz

This is very valuable input! I will look more into this. It sounds like something is wonky around async upgrades.

ghost avatar Sep 16 '21 21:09 ghost

If you do async stuff in upgrade handler without calling res.onAborted you must have seen this:

Error: Returning from a request handler without responding or attaching an abort handler is forbidden!

ghost avatar Sep 16 '21 21:09 ghost

Ok, changed my async code to "open" event. Thanks!

amunhoz avatar Sep 21 '21 21:09 amunhoz

Please keep this issue open. This is definitely a bug in the library and should never happen.

ghost avatar Sep 21 '21 22:09 ghost

I also met this issue,but I am using the C++ version

mc373906408 avatar Jun 22 '22 10:06 mc373906408

Do you use more than 1 thread in your app?

ghost avatar Jun 22 '22 10:06 ghost

Do you use more than 1 thread in your app?

Yes,I have to maintain live555 thread

mc373906408 avatar Jun 22 '22 10:06 mc373906408

How do you communicate between those other threads and uWS? Do you call into uWS from multiple different threads?

ghost avatar Jun 22 '22 10:06 ghost

How do you communicate between those other threads and uWS? Do you call into uWS from multiple different threads?

if i lock before “uWS::App::publish”,Is it possible to fix this situation

mc373906408 avatar Jun 22 '22 10:06 mc373906408

How do you communicate between those other threads and uWS? Do you call into uWS from multiple different threads?

Yes

mc373906408 avatar Jun 22 '22 10:06 mc373906408

uWS is singleton class

mc373906408 avatar Jun 22 '22 10:06 mc373906408

https://github.com/uNetworking/uWebSockets/blob/master/misc/READMORE.md#threading

ghost avatar Jun 22 '22 10:06 ghost

Loop:defer How can I use it, is there an example?

mc373906408 avatar Jun 22 '22 10:06 mc373906408

I think you have enough hints, you can study it on your own. It's impossible for me to guide you exactly as I don't really know anything about your app

ghost avatar Jun 22 '22 10:06 ghost