RcppParallel icon indicating copy to clipboard operation
RcppParallel copied to clipboard

TBB autodetection was switched off

Open Enchufa2 opened this issue 1 year ago • 5 comments

I didn't detect this because I had the old workaround in place, but after #178, I suppose that 640468165c05611ba81d76b757fd3f195a93b285 broke this?

Enchufa2 avatar May 22 '24 14:05 Enchufa2

Most likely a bug in the updated configure script? Can you confirm where libtbb is installed / available on the system you're testing with?

kevinushey avatar May 22 '24 17:05 kevinushey

We have:

$ rpm -ql tbb-devel
/usr/include/oneapi
/usr/include/oneapi/tbb
/usr/include/oneapi/tbb.h
/usr/include/oneapi/tbb/blocked_range.h
/usr/include/oneapi/tbb/blocked_range2d.h
/usr/include/oneapi/tbb/blocked_range3d.h
/usr/include/oneapi/tbb/blocked_rangeNd.h
/usr/include/oneapi/tbb/cache_aligned_allocator.h
/usr/include/oneapi/tbb/collaborative_call_once.h
/usr/include/oneapi/tbb/combinable.h
/usr/include/oneapi/tbb/concurrent_hash_map.h
/usr/include/oneapi/tbb/concurrent_lru_cache.h
/usr/include/oneapi/tbb/concurrent_map.h
/usr/include/oneapi/tbb/concurrent_priority_queue.h
/usr/include/oneapi/tbb/concurrent_queue.h
/usr/include/oneapi/tbb/concurrent_set.h
/usr/include/oneapi/tbb/concurrent_unordered_map.h
/usr/include/oneapi/tbb/concurrent_unordered_set.h
/usr/include/oneapi/tbb/concurrent_vector.h
/usr/include/oneapi/tbb/detail
/usr/include/oneapi/tbb/detail/_aggregator.h
/usr/include/oneapi/tbb/detail/_aligned_space.h
/usr/include/oneapi/tbb/detail/_allocator_traits.h
/usr/include/oneapi/tbb/detail/_assert.h
/usr/include/oneapi/tbb/detail/_attach.h
/usr/include/oneapi/tbb/detail/_concurrent_queue_base.h
/usr/include/oneapi/tbb/detail/_concurrent_skip_list.h
/usr/include/oneapi/tbb/detail/_concurrent_unordered_base.h
/usr/include/oneapi/tbb/detail/_config.h
/usr/include/oneapi/tbb/detail/_containers_helpers.h
/usr/include/oneapi/tbb/detail/_exception.h
/usr/include/oneapi/tbb/detail/_export.h
/usr/include/oneapi/tbb/detail/_flow_graph_body_impl.h
/usr/include/oneapi/tbb/detail/_flow_graph_cache_impl.h
/usr/include/oneapi/tbb/detail/_flow_graph_impl.h
/usr/include/oneapi/tbb/detail/_flow_graph_indexer_impl.h
/usr/include/oneapi/tbb/detail/_flow_graph_item_buffer_impl.h
/usr/include/oneapi/tbb/detail/_flow_graph_join_impl.h
/usr/include/oneapi/tbb/detail/_flow_graph_node_impl.h
/usr/include/oneapi/tbb/detail/_flow_graph_node_set_impl.h
/usr/include/oneapi/tbb/detail/_flow_graph_nodes_deduction.h
/usr/include/oneapi/tbb/detail/_flow_graph_tagged_buffer_impl.h
/usr/include/oneapi/tbb/detail/_flow_graph_trace_impl.h
/usr/include/oneapi/tbb/detail/_flow_graph_types_impl.h
/usr/include/oneapi/tbb/detail/_hash_compare.h
/usr/include/oneapi/tbb/detail/_intrusive_list_node.h
/usr/include/oneapi/tbb/detail/_machine.h
/usr/include/oneapi/tbb/detail/_mutex_common.h
/usr/include/oneapi/tbb/detail/_namespace_injection.h
/usr/include/oneapi/tbb/detail/_node_handle.h
/usr/include/oneapi/tbb/detail/_pipeline_filters.h
/usr/include/oneapi/tbb/detail/_pipeline_filters_deduction.h
/usr/include/oneapi/tbb/detail/_range_common.h
/usr/include/oneapi/tbb/detail/_rtm_mutex.h
/usr/include/oneapi/tbb/detail/_rtm_rw_mutex.h
/usr/include/oneapi/tbb/detail/_scoped_lock.h
/usr/include/oneapi/tbb/detail/_segment_table.h
/usr/include/oneapi/tbb/detail/_small_object_pool.h
/usr/include/oneapi/tbb/detail/_string_resource.h
/usr/include/oneapi/tbb/detail/_task.h
/usr/include/oneapi/tbb/detail/_task_handle.h
/usr/include/oneapi/tbb/detail/_template_helpers.h
/usr/include/oneapi/tbb/detail/_utils.h
/usr/include/oneapi/tbb/detail/_waitable_atomic.h
/usr/include/oneapi/tbb/enumerable_thread_specific.h
/usr/include/oneapi/tbb/flow_graph.h
/usr/include/oneapi/tbb/flow_graph_abstractions.h
/usr/include/oneapi/tbb/global_control.h
/usr/include/oneapi/tbb/info.h
/usr/include/oneapi/tbb/memory_pool.h
/usr/include/oneapi/tbb/mutex.h
/usr/include/oneapi/tbb/null_mutex.h
/usr/include/oneapi/tbb/null_rw_mutex.h
/usr/include/oneapi/tbb/parallel_for.h
/usr/include/oneapi/tbb/parallel_for_each.h
/usr/include/oneapi/tbb/parallel_invoke.h
/usr/include/oneapi/tbb/parallel_pipeline.h
/usr/include/oneapi/tbb/parallel_reduce.h
/usr/include/oneapi/tbb/parallel_scan.h
/usr/include/oneapi/tbb/parallel_sort.h
/usr/include/oneapi/tbb/partitioner.h
/usr/include/oneapi/tbb/profiling.h
/usr/include/oneapi/tbb/queuing_mutex.h
/usr/include/oneapi/tbb/queuing_rw_mutex.h
/usr/include/oneapi/tbb/rw_mutex.h
/usr/include/oneapi/tbb/scalable_allocator.h
/usr/include/oneapi/tbb/spin_mutex.h
/usr/include/oneapi/tbb/spin_rw_mutex.h
/usr/include/oneapi/tbb/task.h
/usr/include/oneapi/tbb/task_arena.h
/usr/include/oneapi/tbb/task_group.h
/usr/include/oneapi/tbb/task_scheduler_observer.h
/usr/include/oneapi/tbb/tbb_allocator.h
/usr/include/oneapi/tbb/tbbmalloc_proxy.h
/usr/include/oneapi/tbb/tick_count.h
/usr/include/oneapi/tbb/version.h
/usr/include/tbb
/usr/include/tbb/blocked_range.h
/usr/include/tbb/blocked_range2d.h
/usr/include/tbb/blocked_range3d.h
/usr/include/tbb/blocked_rangeNd.h
/usr/include/tbb/cache_aligned_allocator.h
/usr/include/tbb/collaborative_call_once.h
/usr/include/tbb/combinable.h
/usr/include/tbb/concurrent_hash_map.h
/usr/include/tbb/concurrent_lru_cache.h
/usr/include/tbb/concurrent_map.h
/usr/include/tbb/concurrent_priority_queue.h
/usr/include/tbb/concurrent_queue.h
/usr/include/tbb/concurrent_set.h
/usr/include/tbb/concurrent_unordered_map.h
/usr/include/tbb/concurrent_unordered_set.h
/usr/include/tbb/concurrent_vector.h
/usr/include/tbb/enumerable_thread_specific.h
/usr/include/tbb/flow_graph.h
/usr/include/tbb/flow_graph_abstractions.h
/usr/include/tbb/global_control.h
/usr/include/tbb/info.h
/usr/include/tbb/memory_pool.h
/usr/include/tbb/mutex.h
/usr/include/tbb/null_mutex.h
/usr/include/tbb/null_rw_mutex.h
/usr/include/tbb/parallel_for.h
/usr/include/tbb/parallel_for_each.h
/usr/include/tbb/parallel_invoke.h
/usr/include/tbb/parallel_pipeline.h
/usr/include/tbb/parallel_reduce.h
/usr/include/tbb/parallel_scan.h
/usr/include/tbb/parallel_sort.h
/usr/include/tbb/partitioner.h
/usr/include/tbb/profiling.h
/usr/include/tbb/queuing_mutex.h
/usr/include/tbb/queuing_rw_mutex.h
/usr/include/tbb/rw_mutex.h
/usr/include/tbb/scalable_allocator.h
/usr/include/tbb/spin_mutex.h
/usr/include/tbb/spin_rw_mutex.h
/usr/include/tbb/task.h
/usr/include/tbb/task_arena.h
/usr/include/tbb/task_group.h
/usr/include/tbb/task_scheduler_observer.h
/usr/include/tbb/tbb.h
/usr/include/tbb/tbb_allocator.h
/usr/include/tbb/tbbmalloc_proxy.h
/usr/include/tbb/tick_count.h
/usr/include/tbb/version.h
/usr/lib64/cmake/TBB
/usr/lib64/cmake/TBB/TBBConfig.cmake
/usr/lib64/cmake/TBB/TBBConfigVersion.cmake
/usr/lib64/cmake/TBB/TBBTargets-relwithdebinfo.cmake
/usr/lib64/cmake/TBB/TBBTargets.cmake
/usr/lib64/libirml.so
/usr/lib64/libtbb.so
/usr/lib64/libtbbbind_2_5.so
/usr/lib64/libtbbmalloc.so
/usr/lib64/libtbbmalloc_proxy.so
/usr/lib64/pkgconfig/tbb.pc
/usr/lib64/pkgconfig/tbbmalloc.pc
/usr/lib64/pkgconfig/tbbmalloc_proxy.pc
/usr/share/doc/tbb-devel
/usr/share/doc/tbb-devel/README.md

Enchufa2 avatar May 22 '24 19:05 Enchufa2

Well, the issue is that autodetection was disabled: https://github.com/RcppCore/RcppParallel/commit/d4315220e29f3e48cdc993f9c6e6cfbc605c88a4. Do you remember the reason?

Enchufa2 avatar May 22 '24 20:05 Enchufa2

And apart from this, I think that the issue with stan-dev/rstan/issues/1041 can be solved here much easily. The issue is that this part:

https://github.com/RcppCore/RcppParallel/blob/8ace3198c420a3d63e7c0cb06795ea8e17b304c1/R/tbb.R#L61-L73

~does not take into account if the system tbb is used via autodetection~. If this chunk adds the -DTBB_INTERFACE_NEW flag in this case, then rstan and family will compile just fine if I'm not mistaken.

EDIT: Sorry, I misread that: the TBB_INC variable is defined in the package namespace when autodetection is enabled. So it should work... The only point is to reenable autodetection.

Enchufa2 avatar May 23 '24 10:05 Enchufa2

Well, the issue is that autodetection was disabled: https://github.com/RcppCore/RcppParallel/commit/d4315220e29f3e48cdc993f9c6e6cfbc605c88a4. Do you remember the reason?

I do not -- it would've been helpful if I had actually documented why somewhere. 😞 My guess is that this was somehow CRAN-oriented; e.g. I wanted to make sure we always used the bundled TBB on CRAN, but even if that's true there's probably a better way to handle that.

kevinushey avatar Jul 03 '24 05:07 kevinushey