Win: crash on export when using darkframe

[masc4ii]

When using darkframe feature and exporting a batch, after some clips the app crashes. This happens until now on Windows only, but not on all Windows. Debugging brings the following (thx to 50mm1200s):

QTCreator points me to line 361 of the file amaze_demosaic.c (on debayer folder), after segfaulting. memset(nyquist, 0, sizeof(char)*TS*TSH); He also has two warnings (line 364 and 366) saying: "use of GNU statement expression extension".

Using bilinear demosaicing, it segfaulted on line 411 of video_mlv.c memcpy(outputFrame, video->rgb_raw_current_frame, frame_size);

And warnings on lines 404, 408 and 409: "implicit conversion changes signedness: 'int' to 'unsigned int'"

On line 409 it also says: implicit conversion loses integer precision: 'uint64_t' (aka 'unsigned long long') to 'int'

The same happens using one thread ("if (1)" on debayer.c), with both bilinear and AMaZE. There's a strage warning on application output, though:

ReturnHr(338) tid(23b8) 8007000F The system could not find the specified unit.
onecoreuap\shell\windows.storage\regfldr.cpp(1239)\windows.storage.dll!7666ED09: (caller: 7666C430)

Using LMSSE and IGV, the debugger doesn't seem to be able to find the issue. It just crashs: "The program has unexpectedly finished. The process was ended forcefully". And then points me to the debug folder.

[bouncyball-git]

Great progress. I wish I had more time :(

[masc4ii]

More information from 50mm1200s: https://www.magiclantern.fm/forum/index.php?topic=20025.msg204788#msg204788

[masc4ii]

I now could reproduce an error exporting with darkframe and trace back. If it crashes, it crashes always with a framebuffer address = 0. This means again (same as in caching bug) that malloc() is not working correctly (or does not allocate memory) on Windows. In startPipe() the malloc on frameBuffer does not work, in openMlv() (MainWindow) malloc on m_pRawFrame does not work, and in getMlvProcessedFrame16() malloc on unprocessed_frame does not work. Whenever processing code writes on one of these variables, app crashes. But WHY malloc returns 0 on Windows (only)? WTF??? There are 6GB RAM unused on my system and we try to allocate 5MB... Same problem here #40 .

[bouncyball-git]

This issue only affects exporting right? And win32 caching issue is affects playback as well. Edit: and both are windows only issues right?

[masc4ii]

As far as I could reproduce, yes. Yes. And Yes. Edit: some hours later and I can't reproduce it again... 😝

[masc4ii]

Suggestion: how would it be to try out VisualC++ compiler?! Maybe this one works better?!

[bouncyball-git]

Yes I thought about it...

[masc4ii]

I tried it yesterday evening... looks like a lot of work: MSVC does not know pthread, restrict, inline, -fopenmp, ... I got hundreds of compiler errors.

[bouncyball-git]

Yes it needs -mthreads and lots of other stuff.

[masc4ii]

Sounds not good. That means implmenting a lot of special code just for trying another compiler...

[bouncyball-git]

BTW MLVFS for windows compiles with VC and it uses multythreading there.

[bouncyball-git]

Dmilligan and especially g3gg0 have done good job for supporting VC.

[bouncyball-git]

It spits out a lot of warnings but compiles and runs smoothly.

[masc4ii]

I checked MLVApp with Valgrind, exporting some clips with darkframe on OSX 10.9.5. The most "critical" I found is this:

==616== Conditional jump or move depends on uninitialised value(s)
==616==    at 0x7A20: strrchr (in /usr/local/Cellar/valgrind/3.13.0/lib/valgrind/vgpreload_memcheck-amd64-darwin.so)
==616==    by 0x1000A3E1D: load_mapp (video_mlv.c:648)
==616==    by 0x1000A01AA: openMlvClip (video_mlv.c:1223)
==616==    by 0x1000A0073: initMlvObjectWithClip (video_mlv.c:475)
==616==    by 0x10003C91F: MainWindow::openMlvForPreview(QString) (MainWindow.cpp:462)
==616==    by 0x100036EBB: MainWindow::importNewMlv(QString) (MainWindow.cpp:435)

==616== Use of uninitialised value of size 8
==616==    at 0x362A285: _platform_memmove$VARIANT$Merom (in /usr/lib/system/libsystem_platform.dylib)
==616==    by 0x34A9D70: __memcpy_chk (in /usr/lib/system/libsystem_c.dylib)
==616==    by 0x1000A3E41: load_mapp (video_mlv.c:650)
==616==    by 0x1000A01AA: openMlvClip (video_mlv.c:1223)
==616==    by 0x1000A0073: initMlvObjectWithClip (video_mlv.c:475)
==616==    by 0x10003C91F: MainWindow::openMlvForPreview(QString) (MainWindow.cpp:462)
==616==    by 0x100036EBB: MainWindow::importNewMlv(QString) (MainWindow.cpp:435)

==772==    Address 0x10b21be2e is 2,420,206 bytes inside a block of size 2,420,208 alloc'd
==772==    at 0x6ABF: calloc (in /usr/local/Cellar/valgrind/3.13.0/lib/valgrind/vgpreload_memcheck-amd64-darwin.so)
==772==    by 0x100103400: df_load_ext (darkframe.c:90)
==772==    by 0x1001030DD: df_validate (darkframe.c:236)
==772==    by 0x1000A9004: llrpValidateExtDarkFrame (llrawproc.c:545)
==772==    by 0x10005DC06: MainWindow::on_lineEditDarkFrameFile_textChanged(QString const&) (MainWindow.cpp:6329)

So nothing really bad I think. Note: all this happens on loading the files into MLVApp. There is no Valgrind output at all while exporting! And all other stuff Valgrind is talking about is related to Qt and Cocoa, but here we can't do anything (and I think it does not change anything for this issue).

[bouncyball-git]

1st 2 are not a real problem, could you send me the full log before the 3rd one?

[bouncyball-git]

it seems that something is trying to write more than 2 bytes starting from 2,420,206nth byte in the buffer. However it could be false alarm ;)

[masc4ii]

could you send me the full log before the 3rd one?

That was all... no more info at this point. There was only Qt and Cocoa stuff before and after.

[bouncyball-git]

ok