darktable
darktable copied to clipboard
darktable-org
libraw: update to latest tag, support R7 and R10
after integrating this pr in current master via git pull -f --no-edit https://github.com/darktable-org/darktable.git pull/12124/head an osx build no longer opens cr3 files. dt freezes
Process 18876 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
frame #0: 0x00000001002f0923 libomp.dylib`__kmp_wait_4 + 211
libomp.dylib`__kmp_wait_4:
-> 0x1002f0923 <+211>: leaq 0x3dc9e(%rip), %rax ; __kmp_use_yield
0x1002f092a <+218>: movl (%rax), %eax
0x1002f092c <+220>: leal -0x1(%rax), %ecx
0x1002f092f <+223>: cmpl $0x1, %ecx
Target 0: (darktable) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
* frame #0: 0x00000001002f0923 libomp.dylib`__kmp_wait_4 + 211
frame #1: 0x00000001002f83fe libomp.dylib`int __kmp_acquire_queuing_lock_timed_template<false>(kmp_queuing_lock*, int) + 494
frame #2: 0x00000001002fd6f9 libomp.dylib`__kmp_acquire_queuing_lock(kmp_user_lock*, int) + 9
frame #3: 0x00000001002ab53a libomp.dylib`__kmpc_critical_with_hint + 1258
frame #4: 0x0000000100ba2934 libdarktable.dylib`libraw_memmgr::mem_ptr(this=0x00007ff7bfefda50, ptr=0x000000011d125500) at libraw_alloc.h:92:1 [opt]
frame #5: 0x0000000100b02130 libdarktable.dylib`crxReadImageHeaders(crx_data_header_t*, CrxImage*, unsigned char*, int) [inlined] libraw_memmgr::calloc(this=0x00007ff7bfefda50, sz=1) at libraw_alloc.h:57:5 [opt]
frame #6: 0x0000000100b02111 libdarktable.dylib`crxReadImageHeaders(hdr=0x00007ff7bfefd970, img=0x00007ff7bfefd9f8, mdatPtr="\xff\U00000001", mdatHdrSize=112) at crx.cpp:2321:20 [opt]
frame #7: 0x0000000100b03280 libdarktable.dylib`crxSetupImageData(hdr=<unavailable>, img=<unavailable>, outBuf=<unavailable>, mdatOffset=<unavailable>, mdatSize=<unavailable>, mdatHdrPtr=<unavailable>, mdatHdrSize=<no summary available>) at crx.cpp:2664:10 [opt] [artificial]
frame #8: 0x0000000100b03917 libdarktable.dylib`LibRaw::crxLoadRaw(this=0x000000011ee19000) at crx.cpp:2778:7 [opt]
frame #9: 0x0000000100b281ec libdarktable.dylib`LibRaw::unpack(this=0x000000011ee19000) at unpack.cpp:400:7 [opt]
frame #10: 0x0000000100a93063 libdarktable.dylib`dt_imageio_open_libraw(img=0x00007ff7bfefe040, filename="/Users/martinstraeten/Downloads/2O3A2490.CR3", mbuf=0x00007ff7bfefe640) at imageio_libraw.c:254:16 [opt]
frame #11: 0x0000000100903354 libdarktable.dylib`dt_imageio_open(img=0x00007ff7bfefe040, filename="/Users/martinstraeten/Downloads/2O3A2490.CR3", buf=0x00007ff7bfefe640) at imageio.c:1273:11 [opt]
frame #12: 0x00000001009306c7 libdarktable.dylib`dt_mipmap_cache_get_with_caller(cache=<unavailable>, buf=0x00007ff7bfefe640, imgid=28416, mip=DT_MIPMAP_FULL, flags=<unavailable>, mode='r', file="/Users/martinstraeten/src/darktable/src/develop/develop.c", line=656) at mipmap_cache.c:781:35 [opt]
frame #13: 0x00000001009a0aad libdarktable.dylib`dt_dev_load_image [inlined] _dt_dev_load_raw(dev=0x00000001091b5400, imgid=28416) at develop.c:656:3 [opt]
frame #14: 0x00000001009a0a18 libdarktable.dylib`dt_dev_load_image(dev=0x00000001091b5400, imgid=28416) at develop.c:710:3 [opt]
frame #15: 0x0000000116017ec0 libdarkroom.so`enter(self=0x000000010a9db5f0) at darkroom.c:3033:3 [opt]
frame #16: 0x0000000100a8b2f5 libdarktable.dylib`dt_view_manager_switch_by_view(vm=<unavailable>, nv=0x000000010a9db5f0) at view.c:398:23 [opt]
frame #17: 0x0000000100a8ac3a libdarktable.dylib`dt_view_manager_switch(vm=<unavailable>, view_name=<unavailable>) at view.c:236:10 [opt] [artificial]
frame #18: 0x0000000100a3392e libdarktable.dylib`_event_button_press(widget=<unavailable>, event=0x00000001095e75e0, user_data=0x0000600003334140) at thumbtable.c:1075:5 [opt]
frame #19: 0x0000000101933e82 libgtk-3.0.dylib`_gtk_marshal_BOOLEAN__BOXED + 97
frame #20: 0x0000000100746716 libgobject-2.0.0.dylib`g_closure_invoke + 192
frame #21: 0x000000010075a96b libgobject-2.0.0.dylib`signal_emit_unlocked_R + 1472
frame #22: 0x000000010075b658 libgobject-2.0.0.dylib`g_signal_emit_valist + 1911
frame #23: 0x000000010075bd28 libgobject-2.0.0.dylib`g_signal_emit + 120
frame #24: 0x00000001018f7c19 libgtk-3.0.dylib`gtk_widget_event_internal + 253
frame #25: 0x00000001017d904e libgtk-3.0.dylib`propagate_event + 369
frame #26: 0x00000001017d84db libgtk-3.0.dylib`gtk_main_do_event + 1077
frame #27: 0x000000010040b955 libgdk-3.0.dylib`_gdk_event_emit + 49
frame #28: 0x000000010042f783 libgdk-3.0.dylib`gdk_event_dispatch + 50
frame #29: 0x000000010055e5e4 libglib-2.0.0.dylib`g_main_context_dispatch + 257
frame #30: 0x000000010055e8ca libglib-2.0.0.dylib`g_main_context_iterate + 431
frame #31: 0x000000010055eb32 libglib-2.0.0.dylib`g_main_loop_run + 250
frame #32: 0x00000001017d7f72 libgtk-3.0.dylib`gtk_main + 74
frame #33: 0x0000000100a4a1ea libdarktable.dylib`dt_gui_gtk_run(gui=<unavailable>) at gtk.c:1275:3 [opt]
frame #34: 0x000000010000ff4e darktable`main(argc=<unavailable>, argv=<unavailable>) at main.c:94:3 [opt]
frame #35: 0x000000010002151e dyld`start + 462
(lldb) up
frame #1: 0x00000001002f83fe libomp.dylib`int __kmp_acquire_queuing_lock_timed_template<false>(kmp_queuing_lock*, int) + 494
libomp.dylib`__kmp_acquire_queuing_lock_timed_template<false>:
-> 0x1002f83fe <+494>: movl %r14d, 0x160(%r13)
0x1002f8405 <+501>: movq $0x0, 0x168(%r13)
0x1002f8410 <+512>: addq $0x28, %rsp
0x1002f8414 <+516>: popq %rbx
(lldb)
frame #2: 0x00000001002fd6f9 libomp.dylib`__kmp_acquire_queuing_lock(kmp_user_lock*, int) + 9
libomp.dylib`__kmp_acquire_queuing_lock:
-> 0x1002fd6f9 <+9>: movl $0x1, %eax
0x1002fd6fe <+14>: popq %rbp
0x1002fd6ff <+15>: retq
libomp.dylib`__kmp_acquire_drdpa_lock:
0x1002fd700 <+0>: pushq %rbp
(lldb)
frame #3: 0x00000001002ab53a libomp.dylib`__kmpc_critical_with_hint + 1258
libomp.dylib`__kmpc_critical_with_hint:
-> 0x1002ab53a <+1258>: leaq 0x85eef(%rip), %rax ; __kmp_itt_sync_acquired_ptr__3_0
0x1002ab541 <+1265>: movq (%rax), %rax
0x1002ab544 <+1268>: testq %rax, %rax
0x1002ab547 <+1271>: je 0x1002ab54e ; <+1278>
(lldb)
libdarktable.dylib was compiled with optimization - stepping may behave oddly; variables may not be available.
frame #4: 0x0000000100ba2934 libdarktable.dylib`libraw_memmgr::mem_ptr(this=0x00007ff7bfefda50, ptr=0x000000011d125500) at libraw_alloc.h:92:1 [opt]
89 #endif
90
91 #if defined(LIBRAW_USE_OPENMP)
-> 92 #pragma omp critical
93 {
94 #endif
95 if (ptr)
Maybe switch off OpenMP for LibRaw w/ "OR APPLE" here for start? Unfortunately I don't have time to check a Windows build w/ OpenMP right now...
Looks like there are not so insignificant changes between the last snapshot and Beta1, so either there are bugs upstream and/or the independent LibRaw-cmake build recipe is out of date...
did a build on an windows machine - same behaviour. Opening a cr3 file results in a freeze
at least that change doesn't affect libraw building with/without openmp.
current master:
-- ----------------------------------------------------------------------------------
-- Libraw 0.21.0 configuration <http://www.libraw.org>
--
-- Libraw will be compiled with OpenMP support .................. YES
-- Libraw will be compiled with LCMS support .................... YES
-- Libraw will be compiled with example command-line programs ... NO
-- Libraw will be compiled with RedCine codec support ........... YES
-- Libraw will be compiled with DNG deflate codec support ....... YES
-- Libraw will be compiled with DNG lossy codec support ......... YES
-- Libraw will be compiled with RawSpeed support ................ NO
-- Libraw will be compiled with debug message from dcraw ........ NO
-- Libraw will be compiled with Foveon X3F support .............. NO
-- Libraw will be compiled with Raspberry Pi RAW support ........ NO
-- Libraw will be compiled as a static library
-- ----------------------------------------------------------------------------------
with changed line 1045 no change in cmake output, so how to disable openmp just for libraw?
To test, I guess then change the ENABLE_OPENMP default in src/external/LibRaw-cmake/CMakeLists.txt
With a clean build folder? You sure CMake cache is s not throwing a wobbly here?
i start my build process with rm -rf ../build/* in my darktable/build directory
on my mac i tried disabling openmp the brute force way by inserting #undef LIBRAW_USE_OPENMP into src/external/LibRaw/libraw/libraw_alloc.h.
Now i'm able to open cr3 files - so it seems the openmp stuff there is a bit strange.
Replacing that file by it's predecessor in the formerly used snapshot also reenables support for cr3, so no need to #undef LIBRAW_USE_OPENMP.
Since the openmp stuff in formerly used snapshot was fine disabling openmp in general doesn't make sense.
will try that tomorrow when i have access to the windows machine
There was essentially no OpenMP sections in the previous snapshot (it was limited to some demosaic and post-processing that was unused by darktable), so building w/ or w/o didn't really make a difference. This, however, in libraw_alloc.h just appeared in the Beta1 yesterday... @LibRaw
doesn't change cmake output and also not the behaviour
At least found a problem w/ the configuration: both the code enablement and the config printout are done on OpenMP_FOUND (which was TRUE from the master darktable project) instead of the sub-project local ENABLE_OPENMP. Will submit a patch for this to LibRaw-cmake upstream...
Thanks! There are already omp critical in crx decoder, so adding additional lock will result into deadlock. Fix expected in a 10 minutes or so :)
This one should fix the problem: https://github.com/LibRaw/LibRaw/commit/90b8e6b5af9974b9b7f3bc6ac2a2611d06b753c5
I also am finding I can not open CR3 files in DT V4. I tried the latest windows build from Bill Ferguson but still no luck. Is it reasonable to expect this issue to be resolved in a bugfix release for V4.0 or should users with CR3 files revert to V3.8? Thanks.
I also am finding I can not open CR3 files in DT V4. I tried the latest windows build from Bill Ferguson but still no luck. Is it reasonable to expect this issue to be resolved in a bugfix release for V4.0 or should users with CR3 files revert to V3.8?
there isn’t an issue in general with cr3 support in v4. It’s just the support for r3,r7,r10. So if you have an issue with cr3 files in general then file an issue for that …
I also am finding I can not open CR3 files in DT V4. I tried the latest windows build from Bill Ferguson but still no luck. Is it reasonable to expect this issue to be resolved in a bugfix release for V4.0 or should users with CR3 files revert to V3.8?
there isn’t an issue in general with cr3 support in v4. It’s just the support for r3,r7,r10. So if you have an issue with cr3 files in general then file an issue for that …
Hi, I have bought (unfortunately??!) an CANON EOS R7 (an excellent camera by the way!) but I don't understand that I have obviously to compile my build of Darktable for myself. I have no clue of compiling anything. Do I really have to buy Lightroom or CaptureOne? My believe in opensource has been shaken. I would be very thankful if there will be an option to download a nightly build of V4.0 with support of R7 (R10 and R3).
You simply have to be patient until this pr is merged. if you don’t understand: even opensource tools needs development effort - if you need stuff fast, then it’s on you to learn how to compile. BTW: that’s very good documented and quite foolsafe ;)
@jackbauer66 Please do some research before. And do not expect miracles. Be patient and help with testing or contribute in some other constructive way.
TIA
Hi, I have bought (unfortunately??!) an CANON EOS R7 (an excellent camera by the way!) but I don't understand that I have obviously to compile my build of Darktable for myself. I have no clue of compiling anything. Do I really have to buy Lightroom or CaptureOne? My believe in opensource has been shaken. I would be very thankful if there will be an option to download a nightly build of V4.0 with support of R7 (R10 and R3).
Having no clue of compiling anything and running a downloaded nightly build do not go well together. I counsel a little patience. The development can't start until Canon allow people to see the CR3 files from the camera, which may not be identical to CR3 files from other cameras with other sensors.
The camera seems quite good at making JPEGs itself, and any CR3 that will produce a much better image will still do so next month. Or year.
Meanwhile you can work around it, by installing the Adobe free download DNG converter which will run under WINE if you are as I am on Linux, or natively if you are on a Mac or running Windows, and will generate a .DNG from each .CR3 in a directory.
The .DNG can then be opened in Darktable (DT4) and go on from there including turning out a 16 bit TIFF that can be imported to the Gimp.
Having no clue of compiling anything and running a downloaded nightly build do not go well together. I counsel a little patience. The development can't start until Canon allow people to see the CR3 files from the camera, which may not be identical to CR3 files from other cameras with other sensors.
thats no issue since latest libraw already supports recent R models. So it's just include the pr mentioned above into a local clone of current master, build it and enjoy cr3 support. (no rocket science and at least working for osx: https://discuss.pixls.us/t/current-osx-build/13213/517)
Now also included parsing of new RF-S lenses that go w/ R7 and R10.
This should be good to go.
