Convert OpenMP parallelization to OneAPI::TBB

[dbs4261]

OpenMP acceleration has been migrated to use oneapi::TBB.

Type

• [ ] Bug fix (non-breaking change which fixes an issue): Fixes #
• [x] New feature (non-breaking change which adds functionality). Resolves #
• [x] Breaking change (fix or feature that would cause existing functionality to not work as expected) Resolves #N/A

Motivation and Context

Many components of Open3D imply an eventual shift away from OpenMP to TBB. This includes some sections where tbb is only used on one platform as 2D loop unrolling isn't supported on Win32. Lastly, by using multiple parallelization paradigms, nested parallelism is problematic. When using some Open3D methods from a TBB context, an OpenMP thread pool is created for each TBB thread.

Checklist:

• [x] I have run python util/check_style.py --apply to apply Open3D code style to my code.
• [ ] This PR changes Open3D behavior or adds new functionality.
  • [ ] Both C++ (Doxygen) and Python (Sphinx / Google style) documentation is updated accordingly.
  • [x] I have added or updated C++ and / or Python unit tests OR included test results (e.g. screenshots or numbers) here.
• [x] I will follow up and update the code if CI fails.
• [x] For fork PRs, I have selected Allow edits from maintainers.

Description

Updated parallel for sections to use tbb::parallel_for. Adapted most loops that performed reductions with either omp reduction clauses or with critical sections to tbb::parallel_reduce implementations. Some of which required custom reduction objects instead of using lambdas. Added an atomic version of the ProgressBar for use with TBB.

There is still work to be done in documentation. This will break any user code that directly uses ParallelForCPU as OpenMP critical sections will no longer work. Additionally, TBB has no approach for setting the maximum number of threads like OpenMP does with OMP_NUM_THREADS. In C++ code a tbb::global_control object could be used, but it is unclear to how to provide that sort of functionality for python users.

[update-docs[bot]]

Thanks for submitting this pull request! The maintainers of this repository would appreciate if you could update the CHANGELOG.md based on your changes.

[dbs4261]

Ok, I ran my tests in my development environment. Guess I should use the docker containers to replicate the CI environment and figure out those tests.

[ssheorey]

Hi @dbs4261 thanks for picking this up!

Possibly fixes #6544

[errissa]

@dbs4261 Thanks for working on this! I just tested this PR on my Mac and got numerous TBB related compilation errors. I tried using the Homebrew version of TBB as well as the "build from source" configuration. There appear to be functions that this PR uses that are missing from the Homebrew and "build from source" versions of TBB on Mac.

I know this PR is still draft but wanted to report what I had found. Please let me know if you need any help testing/diagnosing issues on Mac.

[dbs4261]

Hi @ssheorey this PR likely wont fix that issue as I haven't yet changed how the TBB dependency is being accessed. This is likely also why @errissa is facing issues building on Mac.

@errissa is homebrew pulling the OneAPI version of TBB? If you can provide me with the version of TBB you tried and the compiler errors I can take a look and figure out which version is required and work that into the PR.

[ssheorey]

@dbs4261 yes, you are right about not fixing #6544. We should update to the latest oneTBB as part of this PR to fix that though.

This is the latest version of oneTBB and is available for all platforms on github:

https://github.com/oneapi-src/oneTBB/releases/tag/v2021.11.0

The naming is off - this was released in Nov 2023.

I think this should also resolve @errissa 's issues on macOS.

[dbs4261]

I agree that setting the version requirement for TBB should be part of this PR. Based on the ubuntu failure in CI, its the collaborative_call_once header that is missing. The TBB repo says the header hasnt been modified in 3 years, so I would think that any version that reports 2021+ should be fine. What does Open3D CI currently use for TBB?

[errissa]

@dbs4261 @ssheorey is correct about the oneTBB version. Homebrew's most recent version is 2021.11.0 so if this PR builds successfully against it, it would solve the MacOS issue I experienced.

[dbs4261]

Looks like the minimum version requirement for collaborative_call_once.h is v2021.4.0. Now it looks like we aren't putting version requirements in the find package scripts in 3rdparty/find_dependencies.cmake, this means that error's like @errissa can still happen when using the system library. This raises the question of if I should set the system version requirement to the same version that I am providing in the ExternalProject_add call, or if I should add the newest version but set the system requirement to the minimal version.

[ssheorey]

Hi @dbs4261 , our usual policy is to upgrade to the latest version available, but set minimum version to what is required to make everything work. This helps to "future-proof" the updated code as much as possible by incorporating the latest bugfixes. Official binaries will be built with the latest version, but also allows the library to build on older versions by users.

[ssheorey]

[Notes about linking and binary distribution]

For linking TBB, recommendation is to link dynamically. For C++ binaries and applications, we will distribute TBB DLL along with the Open3D DLL. https://github.com/oneapi-src/oneTBB/issues/646

For Python, TBB libraries are available through PyPI, so we can add these as dependencies to requirements.txt https://community.intel.com/t5/Intel-oneAPI-Threading-Building/How-to-ship-a-package-using-TBB-on-PyPI-manylinux/m-p/1227574