node-mongolian icon indicating copy to clipboard operation
node-mongolian copied to clipboard

Published 20 hours ago • verified publisher icon marcello3d

Error: Buffer is not large enough

Open qharlie opened this issue 12 years ago • 12 comments

I'm getting this error a lot , here is the full stack trace:

Error: Buffer is not large enough. Expected 1745879154, have 36 new Response (/opt/deploy/formhog/node_modules/mongolian/node_modules/buffalo/lib/mongo.js:255:15),Connection. (/opt/deploy/formhog/node_modules/mongolian/lib/server.js:227:28),Connection.EventEmitter.emit (events.js:88:17),Socket. (/opt/deploy/formhog/node_modules/mongolian/lib/connection.js:61:22),Socket.EventEmitter.emit (events.js:88:17),TCP.onread (net.js:397:14)

qharlie avatar Feb 04 '13 15:02 qharlie

I have seen this occasionally as a cascade from a program error/exception. Can you reproduce it?

marcello3d avatar Feb 06 '13 15:02 marcello3d

Ahh thats probably it. Let me try something.

qharlie avatar Feb 06 '13 15:02 qharlie

Yeah I have this problem too, though usually only when I haven't properly handled an error somewhere else. I haven't been able to reliably reproduce it, it just sort of shows up among a bazillion other errors when things go wrong.

awwright avatar Feb 12 '13 14:02 awwright

I've only been able to reproduce it after another error. I'm suspecting there may be a missing var or similar, somewhere. But I can't see anything obviously wrong with the code!

2013-03-14T15:41:51.285Z - Uncaught Exception: Error: Mongo query should have updated 1 documents, but only updated 0
    at lib/transform.HTTPAuto_typeMongoDB_Post.js:178:13
    at mongolian/lib/collection.js:97:13
    at mongolian/lib/util.js:19:13
    at mongolian/lib/db.js:69:25
    at mongolian/lib/util.js:19:13
    at MongolianCursor.next (mongolian/lib/cursor.js:167:9)
    at mongolian/lib/cursor.js:171:18
    at mongolian/lib/util.js:19:13
    at mongolian/lib/cursor.js:125:9
    at mongolian/lib/util.js:19:13
2013-03-14T15:42:02.393Z - Uncaught Exception: Error: Buffer is not large enough. Expected 1159, have 101
    at new Response (buffalo/lib/mongo.js:255:15)
    at Connection.<anonymous> (mongolian/lib/server.js:227:28)
    at Connection.EventEmitter.emit (events.js:94:17)
    at Socket.<anonymous> (mongolian/lib/connection.js:61:22)
    at Socket.EventEmitter.emit (events.js:94:17)
    at Socket.<anonymous> (_stream_readable.js:648:5)
    at Socket.EventEmitter.emit (events.js:91:17)
    at emitReadable (_stream_readable.js:358:10)
    at onread (_stream_readable.js:343:7)
    at ReadableState.onread (_stream_readable.js:64:5)
2013-03-14T15:42:04.999Z - Uncaught Exception: Error: Buffer is not large enough. Expected 1701080942, have 101
    at new Response (buffalo/lib/mongo.js:255:15)
    at Connection.<anonymous> (mongolian/lib/server.js:227:28)
    at Connection.EventEmitter.emit (events.js:94:17)
    at Socket.<anonymous> (mongolian/lib/connection.js:61:22)
    at Socket.EventEmitter.emit (events.js:94:17)
    at Socket.<anonymous> (_stream_readable.js:648:5)
    at Socket.EventEmitter.emit (events.js:91:17)
    at emitReadable (_stream_readable.js:358:10)
    at onread (_stream_readable.js:343:7)
    at ReadableState.onread (_stream_readable.js:64:5)

awwright avatar Mar 14 '13 16:03 awwright

If only half a message of the stream were to come in (which is perfectly legal), what would happen here? https://github.com/marcello3d/node-mongolian/blob/f1a6bdc547db1021321acc869d530ac05dda24d3/lib/connection.js#L59-L79

It appears that there's no test to make sure we have a complete message before emitting a "have packet" event.

awwright avatar Mar 15 '13 00:03 awwright

The while loop condition should handle that case.

marcello3d avatar Mar 15 '13 00:03 marcello3d

So this probably is the problem then?

I'm currently re-writing the file to avoid Buffer re-allocations, among other things

awwright avatar Mar 15 '13 02:03 awwright

The solution might be to move the emitting on line 61 to the bottom of that block. That way if the message handler throws an exception, the buffer is in a consistent state.

marcello3d avatar Mar 15 '13 03:03 marcello3d

That might be the case for the second and beyond errors ("Expected 1701080942") but the first one seems to simply be an incomplete packet, with no prior error having been thrown.

Oh wait...

awwright avatar Mar 15 '13 04:03 awwright

Yeah, that's exactly the problem: whenever an Exception is thrown in an emitter hook, it interrupts the stream handling and an entire segment goes unparsed. So, how should errors be handled? Hmmm

awwright avatar Mar 15 '13 05:03 awwright

Should it really be disconnecting on error by default: https://github.com/marcello3d/node-mongolian/blob/f1a6bdc547db1021321acc869d530ac05dda24d3/lib/server.js#L212-L215 ?

Here's my file that seems to fix: https://gist.github.com/Acubed/5167758 And available in commit Acubed/node-mongolian@2b48848c31cc80c4d0b18483cd401eeea1b9033d

awwright avatar Mar 15 '13 05:03 awwright

Perhaps the event should be called with process.nextTick? That would isolate any errors from affecting the main loop. I'm not so sure if it should disconnect from the database, but may be that too.

awwright avatar Mar 16 '13 02:03 awwright