SEGV in mongo_cursor_destroy

[superarhow]

Thank you all for the great works.

Recently our program crashed when it tries to retrieve very huge data from mongodb server, here's the back trace: (gdb) where 0 0x00002aaab24c3385 in mongo_cursor_destroy (cursor=0x10184970) at libmongo/mongo.c:1390 1 0x00002aaab24c3620 in mongo_cursor_get_more (cursor=0x10184970) at libmongo/mongo.c:1234 2 0x00002aaab24c3b38 in mongo_cursor_next (cursor=0x10184970) at libmongo/mongo.c:1362 3 0x00002aaab24cd84d in rmongo_cursor_next (cursor=) at api_mongo.c:309 4 0x00002ba38d25f4ec in do_dotcall (call=0x10f074f8, op=, args=, env=0x1) at dotcode.c:581 5 0x00002ba38d294cae in Rf_eval (e=0x10f074f8, rho=0x11423440) at eval.c:656 6 0x00002ba38d29a53f in Rf_applyClosure (call=0x105410f0, op=0x10f079c8, arglist=0x114233d0, rho=0x10535980, suppliedenv=0xf668118) at eval.c:1043 7 0x00002ba38d29481f in Rf_eval (e=0x105410f0, rho=0x10535980) at eval.c:675 8 0x00002ba38d2985d7 in do_while (call=0x10542be8, op=0xf6345f8, args=0x105410b8, rho=0x10535980) at eval.c:1556 9 0x00002ba38d294a74 in Rf_eval (e=0x10542be8, rho=0x10535980) at eval.c:628 10 0x00002ba38d296924 in do_begin (call=0x10542750, op=0xf636e20, args=0x10542bb0, rho=0x10535980) at eval.c:1632 11 0x00002ba38d294a74 in Rf_eval (e=0x10542750, rho=0x10535980) at eval.c:628 12 0x00002ba38d294a74 in Rf_eval (e=0x10543600, rho=0x10535980) at eval.c:628 13 0x00002ba38d296924 in do_begin (call=0x10554e78, op=0xf636e20, args=0x105435c8, rho=0x10535980) at eval.c:1632 14 0x00002ba38d294a74 in Rf_eval (e=0x10554e78, rho=0x10535980) at eval.c:628 15 0x00002ba38d29a53f in Rf_applyClosure (call=0x1055d6a0, op=0x1055c960, arglist=0x105382f8, rho=0xf6680e0, suppliedenv=0xf668118) at eval.c:1043 16 0x00002ba38d29481f in Rf_eval (e=0x1055d6a0, rho=0xf6680e0) at eval.c:675 17 0x00002ba38d297375 in do_set (call=0x1055d748, op=0xf637018, args=0x1055d710, rho=0xf6680e0) at eval.c:2029 18 0x00002ba38d294a74 in Rf_eval (e=0x1055d748, rho=0xf6680e0) at eval.c:628 19 0x00002ba38d2bbb49 in Rf_ReplIteration (rho=0xf6680e0, savestack=0, browselevel=, state=0x7fffe5df2970) at main.c:257 20 0x00002ba38d2bbf68 in R_ReplConsole (rho=0xf6680e0, savestack=0, browselevel=0) at main.c:306 21 0x00002ba38d2bc434 in run_Rmainloop () at main.c:998 22 0x0000000000bbe91b in r::session::runEmbeddedR(core::FilePath const&, core::FilePath const&, bool, bool, SA_TYPE, r::session::Callbacks const&, r::session::InternalCallbacks*) () 23 0x0000000000b96641 in r::session::run(r::session::ROptions const&, r::session::RCallbacks const&) () 24 0x00000000006655d8 in main () (gdb) p *cursor $4 = {reply = 0x2aaab26d7010, conn = 0x10060e90, ns = 0x10fa42f0 "PRV.HDS_PID_HDS_SID__auto_k1mEvent_Rel", flags = 3, seen = 12772, current = { data = 0x2aaab2ad6f33 <Address 0x2aaab2ad6f33 out of bounds>, cur = 0x0, dataSize = 0, finished = 1, stack = {0 <repeats 32 times>}, stackPos = 0, err = 0, errstr = 0x0}, err = MONGO_CURSOR_EXHAUSTED, query = 0x7fffe5df0990, fields = 0x10b95690, options = 0, limit = 0, skip = 0} (gdb) p *cursor->reply Cannot access memory at address 0x2aaab26d7010 (gdb)

It can reproduce with a high possibility. I noticed that was a mongo-c-driver bug in v0.7.1 and was fixed at v0.8 (https://github.com/mongodb/mongo-c-driver/commits/v0.8) I tried to use the code in v0.8 and created a patch to fix it, but I think it is better to use the latest mongo-c-driver. Thank you again.

10/6 edit to remove "# number" to avoid search engine's problem:)

[schmidb]

Hi, using any mongo-c driver version higher than v.0.7.1 requires a complete re-engineering of the rmongodb package. I am not planing to do that.

[jeroen]

@schmidb what exactly has changed in 0.7.2? How much work would it be to upgrade to 1.0?

[schmidb]

90% changes and a lot of new concepts. It means more or less re-implementing rmongodb.

On 17.09.2014, at 12:07, Jeroen Ooms [email protected] wrote:

@schmidb what exactly has changed in 0.7.2? How much work would it be to upgrade to 1.0?

— Reply to this email directly or view it on GitHub.

[jeroen]

That's a shame. But the old driver works fine with recent versions of mongodb?

[schmidb]

yes, fully compatible. But there are missing some of the new features.

On 17.09.2014, at 12:43, Jeroen Ooms [email protected] wrote:

That's a shame. But the old driver works fine with recent versions of mongodb?

— Reply to this email directly or view it on GitHub.

[dselivanov]

@superarhow, can you remove #(NUMBER) from your post? Github referenced this issue to issues wtih id from 0 to 24 :-)

[dselivanov]

I see some API change starting from 0.8.0 version. But seems it is not full API break, like it is in 0.9+. I think we can try rebase rmongodb to 0.8.1.

[jeroen]

FYI I started working on a full rewrite at https://github.com/jeroenooms/mongolite

[dselivanov]

@jeroenooms , great news! I thought about starting new branch based on 1.*+ mongo-c-driver or even mongo-cxx-driver using Rcpp. Prior to my recent contributions to rmongodb I didn't know anything about R's C interface(which is not so easy). I always used Rcpp. So I think new branch or new implementation of R-MongoDB driver using modern C++ interface will be more straightforward. Nevertheless there is still left a great deal of work to do.

[dselivanov]

branch for 0.8.1 version of mongo-c-driver: https://github.com/mongosoup/rmongodb/tree/rmongodb_2_0