segmentation fault when queryImgID is called with fast=false

[knifenomad]

I'm managing 300k of image files and it crashes when I call

queryImgID(dbId, imgId, numres=28, sketch=0, fast=false)

(gdb) bt
#0 0x00007ffff5febef3 in queryImgDataFiltered (dbId=0, sig1=, sig2=, sig3=, avgl=, numres=28, sketch=0, bfilter=0x0, colorOnly=false)

at imgSeekLib/imgdb.cpp:715                                                                                                                                                                             

#1 0x00007ffff5fed1ae in queryImgData (dbId=, sig1=, sig2=, sig3=, avgl=, numres=28, sketch=0, colorOnly=false)

at imgSeekLib/imgdb.cpp:768                                                                                                                                                                             

#2 0x00007ffff5fed4bc in queryImgID (dbId=, id=1418, numres=28, sketch=0, colorOnly=) at imgSeekLib/imgdb.cpp:1026 #3 0x00007ffff6003886 in _wrap_queryImgID (args=) at imgSeekLib/imgdb_wrap.cpp:10822 #4 0x00000000004b6569 in PyEval_EvalFrameEx () #5 0x00000000004bcd2d in PyEval_EvalCodeEx () #6 0x0000000000449098 in ?? () #7 0x000000000041ad2a in PyObject_Call () #8 0x00000000004b84b6 in PyEval_EvalFrameEx () #9 0x00000000004bd325 in PyEval_EvalCodeEx () #10 0x0000000000449098 in ?? () #11 0x000000000041ad2a in PyObject_Call () #12 0x00000000004b84b6 in PyEval_EvalFrameEx () #13 0x00000000004bd325 in PyEval_EvalCodeEx () #14 0x00000000004b6a5b in PyEval_EvalFrameEx () #15 0x00000000004bcd2d in PyEval_EvalCodeEx () #16 0x0000000000449098 in ?? () #17 0x000000000041ad2a in PyObject_Call () #18 0x00000000004b84b6 in PyEval_EvalFrameEx () #19 0x00000000004bcd2d in PyEval_EvalCodeEx () #20 0x0000000000448edf in ?? () #21 0x000000000041ad2a in PyObject_Call () #22 0x00000000004b84b6 in PyEval_EvalFrameEx () #23 0x00000000004b6d77 in PyEval_EvalFrameEx () #24 0x00000000004b6d77 in PyEval_EvalFrameEx () #25 0x00000000004b6d77 in PyEval_EvalFrameEx () #26 0x00000000004b6d77 in PyEval_EvalFrameEx () #27 0x00000000004b6d77 in PyEval_EvalFrameEx () #28 0x00000000004b6d77 in PyEval_EvalFrameEx () #29 0x00000000004b6d77 in PyEval_EvalFrameEx () #30 0x00000000004b6d77 in PyEval_EvalFrameEx () #31 0x00000000004b6d77 in PyEval_EvalFrameEx () #32 0x00000000004b6d77 in PyEval_EvalFrameEx () #33 0x00000000004b6d77 in PyEval_EvalFrameEx () #34 0x00000000004bcd2d in PyEval_EvalCodeEx () #35 0x0000000000449098 in ?? () #36 0x000000000041ad2a in PyObject_Call () #37 0x00000000004b84b6 in PyEval_EvalFrameEx () #38 0x00000000004bcd2d in PyEval_EvalCodeEx () #39 0x0000000000449098 in ?? () #40 0x000000000041ad2a in PyObject_Call () #41 0x00000000004b84b6 in PyEval_EvalFrameEx () #42 0x00000000004bcd2d in PyEval_EvalCodeEx ()

---Type to continue, or q to quit--- #43 0x0000000000449098 in ?? () #44 0x000000000041ad2a in PyObject_Call () #45 0x00000000004b84b6 in PyEval_EvalFrameEx () #46 0x00000000004bcd2d in PyEval_EvalCodeEx () #47 0x0000000000449098 in ?? () #48 0x000000000041ad2a in PyObject_Call () #49 0x00000000004b84b6 in PyEval_EvalFrameEx () #50 0x00000000004bcd2d in PyEval_EvalCodeEx () #51 0x00000000004b6a5b in PyEval_EvalFrameEx () #52 0x00000000004b6d77 in PyEval_EvalFrameEx () #53 0x00000000004b6d77 in PyEval_EvalFrameEx () #54 0x00000000004bcd2d in PyEval_EvalCodeEx () #55 0x00000000004b6a5b in PyEval_EvalFrameEx () #56 0x00000000004b6d77 in PyEval_EvalFrameEx () #57 0x00000000004bcd2d in PyEval_EvalCodeEx () #58 0x00000000004bd802 in PyEval_EvalCode () #59 0x00000000004dcc22 in ?? () #60 0x00000000004dd7e4 in PyRun_FileExFlags () #61 0x00000000004b3fe6 in ?? () #62 0x00000000004b6569 in PyEval_EvalFrameEx () #63 0x00000000004bcd2d in PyEval_EvalCodeEx () #64 0x00000000004b6a5b in PyEval_EvalFrameEx () #65 0x00000000004b6d77 in PyEval_EvalFrameEx () #66 0x00000000004bcd2d in PyEval_EvalCodeEx () #67 0x00000000004bd802 in PyEval_EvalCode () #68 0x00000000004dcc22 in ?? () #69 0x00000000004dd7e4 in PyRun_FileExFlags () #70 0x00000000004de2ee in PyRun_SimpleFileExFlags () #71 0x00000000004ee6dd in Py_Main () #72 0x00007ffff699c30d in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6 #73 0x000000000041cb69 in _start ()

[ricardocabral]

what solved this?

[thanigaiv]

I have the same issue on an index with 100K images. Is this issue resolved or is there any workaround. It happens only when sending "fast = false"

[adipasquale]

I am also encountering this segmentation fault issue when running with Fast on Mac OS X 10.8.3 . would you have some advices @ricardocabral ? Overall, would you recommend using imgseek ? you seem to have had quite a few issues with it ...

[ricardocabral]

This doesn't seem solved yet. Investigation pending.

[ricardocabral]

Thanks, you're right! I've applied the fixes to github.

Ricardo http://about.me/rncabral

On Wed, Oct 23, 2013 at 1:57 PM, nicsor [email protected] wrote:

Hi Ricardo, I managed to reproduce the issue and identify the reason. It seems that the issue is in addImageFromImage where you have a case in which you return "ERROR: dbId already in use". The problem is that you delete the element and return, which would leave undefined data in memory and when a query for the id with the issue is made, a segmentation fault occurs. Also, the code in that area had a memory leak of size SigStruct whenever we were on this case.

Please find attached the proposed solution. Also please reply if you agree with this modification.

Best Regards, Nicolae Natea

The changes are under tags /* +23/10/2013 nnatea isk daemon issue - Sharalike / / -23/10/2013 nnatea isk daemon issue - Sharalike */

int addImageFromImage(const int dbId, const long int id, Image * image ) {

..................

transformChar(rchan, gchan, bchan, cdata1, cdata2, cdata3);

DestroyExceptionInfo(&exception);

/* +23/10/2013 nnatea isk daemon issue - Sharalike / //should be created after the check for duplicates #if 0 / -23/10/2013 nnatea isk daemon issue - Sharalike / SigStruct *nsig = new SigStruct(); nsig->id = id; nsig->width = width; nsig->height = height; / +23/10/2013 nnatea isk daemon issue - Sharalike / #endif / -23/10/2013 nnatea isk daemon issue - Sharalike */

if (dbSpace[dbId]->sigs.count(id)) { /* +23/10/2013 nnatea isk daemon issue - Sharalike / // item exists ... but why delete it and return? // either delete it and continue OR not delete it and return #if 0 / -23/10/2013 nnatea isk daemon issue - Sharalike / delete dbSpace[dbId]->sigs[id]; dbSpace[dbId]->sigs.erase(id); / +23/10/2013 nnatea isk daemon issue - Sharalike / #endif / -23/10/2013 nnatea isk daemon issue - Sharalike */

cerr << "ERROR: dbId already in use" << endl;
pthread_mutex_unlock( &mtx );

return 0;

}

/* +23/10/2013 nnatea isk daemon issue - Sharalike / //should be created after the check for duplicates SigStruct *nsig = new SigStruct(); nsig->id = id; nsig->width = width; nsig->height = height; / -23/10/2013 nnatea isk daemon issue - Sharalike */

// insert into sigmap dbSpace[dbId]->sigs[id] = nsig;

.............

}

— Reply to this email directly or view it on GitHubhttps://github.com/ricardocabral/iskdaemon/issues/52#issuecomment-26917507 .

[drevicko]

This is probably related: If I add an image from a file with id set to 0 into an empty (initialised) db:

addImg(dbId, 0, filename)

A subsequent query on id 0 returns an error along the lines of "not in database". A query on id 1, however, returns an empty list (as expected for id 0).

I discovered this after adding 419 images starting at id 0 (up to id 418). The log showed adding with those id's, but I could not query any of them. I'm guessing that if I queried 419, I'd have got the last one?