[doc] Compilation of taichi C-API export library

[PENGUINLIONG]

This issue presents an overview of the Taichi C-API library building process. This is a primer to a more formal documentation but in case of any defects please feel free to leave your commensts here.

NOTE Discussion about common crashes you may easily encounter is attached in the end of this post, you don't want to miss that.

Build Taichi C-API on Windows

For Windows specifically, instead of following the Developer Installation guide on docs.taichi-lang.org and build Taichi with setup.py, I here recommend you simply run the PowerShell script build.ps1 with the following arguments.

./build.ps1 -UserSpace -BuildType Debug

The command will automatically download and unzip LLVM and Clang dependencies to tmp directory for you and install a debuggable version of Taichi in your user home directory.

After the first build, modify _skbuild\win-amd64-3.8\cmake-build\CMakeCache.txt and change TI_WITH_C_API to ON. You might also want to disable the backends you don't need for distribution (by settings TI_WITH_XXX=OFF).

//No help, variable specified on the command line.
-TI_WITH_C_API:UNINITIALIZED=OFF
+TI_WITH_C_API:UNINITIALIZED=ON

Run ./build.ps1 again with the same arguments and Taichi C-API library will be built. In case of a successful build, you should find taichi_c_api.lib in _skbuild\win-amd64-3.8\cmake-install\c_api\lib and there should be a long list of the following in the log:

...
-- Up-to-date: C:/Users/PENGUINLIONG/Repositories/taichi/_skbuild/win-amd64-3.8/cmake-install/c_api/lib/taichi_c_api.lib
-- Up-to-date: C:/Users/PENGUINLIONG/Repositories/taichi/_skbuild/win-amd64-3.8/cmake-install/c_api/bin/taichi_c_api.dll
-- Up-to-date: C:/Users/PENGUINLIONG/Repositories/taichi/_skbuild/win-amd64-3.8/cmake-install/c_api/lib/cmake/taichi_c_api/taichi_c_apiTargets.cmake
-- Up-to-date: C:/Users/PENGUINLIONG/Repositories/taichi/_skbuild/win-amd64-3.8/cmake-install/c_api/lib/cmake/taichi_c_api/taichi_c_apiTargets-debug.cmake
-- Up-to-date: C:/Users/PENGUINLIONG/Repositories/taichi/_skbuild/win-amd64-3.8/cmake-install/c_api/lib/cmake/taichi_c_api/taichi_c_apiConfig.cmake
-- Up-to-date: C:/Users/PENGUINLIONG/Repositories/taichi/_skbuild/win-amd64-3.8/cmake-install/c_api/lib/cmake/taichi_c_api/taichi_c_apiConfigVersion.cmake
-- Up-to-date: C:/Users/PENGUINLIONG/Repositories/taichi/_skbuild/win-amd64-3.8/cmake-install/c_api/include
-- Up-to-date: C:/Users/PENGUINLIONG/Repositories/taichi/_skbuild/win-amd64-3.8/cmake-install/c_api/include/taichi
-- Up-to-date: C:/Users/PENGUINLIONG/Repositories/taichi/_skbuild/win-amd64-3.8/cmake-install/c_api/include/taichi/taichi.h
-- Up-to-date: C:/Users/PENGUINLIONG/Repositories/taichi/_skbuild/win-amd64-3.8/cmake-install/c_api/include/taichi/taichi_core.h
-- Up-to-date: C:/Users/PENGUINLIONG/Repositories/taichi/_skbuild/win-amd64-3.8/cmake-install/c_api/include/taichi/taichi_platform.h
-- Up-to-date: C:/Users/PENGUINLIONG/Repositories/taichi/_skbuild/win-amd64-3.8/cmake-install/c_api/include/taichi/taichi_unity.h
-- Up-to-date: C:/Users/PENGUINLIONG/Repositories/taichi/_skbuild/win-amd64-3.8/cmake-install/c_api/include/taichi/taichi_vulkan.h
...

It tells you where the built libraries are installed. Taking the log above as an example, we say that:

• TAICHI_REPO_DIR = C:/Users/PENGUINLIONG/Repositories/taichi;
• TAICHI_C_API_INSTALL_DIR = C:/Users/PENGUINLIONG/Repositories/taichi/_skbuild/win-amd64-3.8/cmake-install/c_api.

I strongly recommend you to set these paths as environment variables because you would see them everywhere in Taichi's AOT repositories.

Build Taichi C-API on Linux/macOS

Follow the instructions in the Developer Installation guide on docs.taichi-lang.org, but remember to set TI_WITH_C_API=ON.

If you are uncertain whether CMake has cached an unwanted state, please check the generated cache _skbuild\win-amd64-3.8\cmake-build\CMakeCache.txt. You should be able to find something like this:

//No help, variable specified on the command line.
TI_WITH_C_API:UNINITIALIZED=ON

You can then find your libtaichi_c_api.so in /home/penguinliong/Repositories/taichi/_skbuild/linux-x86_64-3.9/cmake-install/c_api/lib. I recommend you to set up environment variables like the following, because they have been used everywhere in Taichi's AOT repositories:

• TAICHI_REPO_DIR = /home/penguinliong/Repositories/taichi;
• TAICHI_C_API_INSTALL_DIR = /home/penguinliong/Repositories/taichi/_skbuild/linux-x86_64-3.9/cmake-install/c_api.

Build Taichi C-API for Android

In PR #5453 I have proposed an example PowerShell script to build an Android .so library on Windows, which is basically:

mkdir build-android-aarch64
cd build-android-aarch64

cmake -DCMAKE_BUILD_TYPE=Release `
    -DCMAKE_INSTALL_PREFIX="${pwd}/../build-android-aarch64/install" `
    -DCMAKE_TOOLCHAIN_FILE="$env:ANDROID_NDK/build/cmake/android.toolchain.cmake" `
    -DCLANG_EXECUTABLE="${pwd}/../tmp/taichi-clang/bin/clang++.exe" `
    -DANDROID_ABI="arm64-v8a" `
    -DANDROID_PLATFORM=android-26 `
    -G "Ninja" `
    -DTI_WITH_CC=OFF `
    -DTI_WITH_CUDA=OFF `
    -DTI_WITH_CUDA_TOOLKIT=OFF `
    -DTI_WITH_C_API=ON `
    -DTI_WITH_DX11=OFF `
    -DTI_WITH_LLVM=OFF `
    -DTI_WITH_METAL=OFF `
    -DTI_WITH_OPENGL=OFF `
    -DTI_WITH_PYTHON=OFF `
    -DTI_WITH_VULKAN=ON `
    ..

cmake --build . -t taichi_c_api
cmake --build . -t install

This is a PowerShell script yet I believe you can easily translate into a Bash one. Remember to replace the clang++ executable path specified in -DCLANG_EXECUTABLE in you have installed clang somewhere else.

In case of a successful build, you can find the built libtaichi_c_api.so and headers in build-android-aarch64\install\c_api.

Link to Taichi C-API Library in a CMake Project

You need the following lines of boilerplate to integrate taichi_c_api into your CMake project:

set(TAICHI_C_API_INSTALL_DIR $ENV{TAICHI_REPO_DIR}/_skbuild/win-amd64-3.8/cmake-install/c_api CACHE PATH "path to taichi c-api installation directory")

find_library(taichi_c_api_LIBRARY taichi_c_api HINTS ${TAICHI_C_API_INSTALL_DIR}/lib REQUIRED NO_CMAKE_FIND_ROOT_PATH)

target_link_libraries(${YOUR_BUILD_TARGET} PRIVATE ${taichi_c_api_LIBRARY})
target_include_directories(${YOUR_BUILD_TARGET} PRIVATE ${${TAICHI_C_API_INSTALL_DIR}/include})

WARNING that the NDK toolchain overrides CMAKE_FIND_ROOT_PATH so your hints in find_library will be ignored. CMake complains that it cannot find taichi_c_api without NO_CMAKE_FIND_ROOT_PATH.

Debugging Projects with Taichi C-API

Taichi C-API currently lacks a mechanism of error reporting and it ought to be fixed real soon. But before that please build your C-API libraries in Debug config for better debug experience.

Appendix I: Common Crashes

Here lies the bodies of poor ancients who spent countless hours battling against those bugs.

SIGSEGV: C-API returns TI_NULL_HANDLE

Where	Why
Any interface that returns a Ti* handle.	Invalid argument(s).

• Ensure non-null handle arguments are not nullptr;
• Ensure allocation sizes are not 0;
• Ensure kernel names and compute graph names you try to get an handle with actually exist;
• Ensure enum and bit mask arguments are in valid ranges.

There might be more information in the logs.

FATAL: bad allocation

Where	Why
ti_load_aot_module	AOT module serialization has broken.

You should rerun the Python script and regenerate the AOT modules.

FATAL Errors on Launch

Where	Why
ti_launch_kernel, ti_launch_compute_graph	Invalid arguments.

• Ensure your argument types matches those in the Python script;
• Ensure your NDArray data types matches those in the Python script;
• Ensure your NDArray shape matches those in the Python script;
• Ensure your list of TiArguments provided sufficient number of arguments referenced by your kernels;
• Ensure your list of TiNamedArguments provided all arguments referenced by your compute graphs.

Please consult the log for more information. In most of the cases your kernel or compute graph arguments are incorrect. This class of fatal errors should become recoverable in the future.

[k-ye]

Cool! Can we include this to https://github.com/taichi-dev/taichi/tree/master/docs ?

[PENGUINLIONG]

Cool! Can we include this to https://github.com/taichi-dev/taichi/tree/master/docs ?

@k-ye I'm gonna leave this as an issue and iterate it for a few days. This doc might be too volatile to become a part of the repository right now. 🫠

[k-ye]

Notes: a few things we should highlight:

• [ ] Use clang to build Taichi on Linux (maybe all?) platforms
• [ ] Give out specific instruction like TAICHI_CMAKE_ARGS="..." python setup.py develop somewhere, so that users can just copy/paste/execute it.
• [ ] Let's ensure TI_WITH_LLVM is no longer a problem

[qiao-bo]

Double checked on Ubuntu 20.04 for building Taichi C-API lib, the following build commands worked for me:

Prerequisite (mainly for build with llvm):

• Use clang-10 instead of gcc:
  • After sudo apt install clang-10, just do export CXX=/usr/bin/clang++-10 to make sure cmake picks up the right compiler.
• Use the pre-built LLVM binary provided as documented here.

Build with llvm:

python setup.py clean && TAICHI_CMAKE_ARGS="-DTI_WITH_C_API:BOOL=ON -DTI_WITH_VULKAN:BOOL=ON -DTI_EXPORT_CORE:BOOL=ON" python setup.py develop

Build without llvm:

python setup.py clean && TAICHI_CMAKE_ARGS="-DTI_WITH_C_API:BOOL=ON -DTI_WITH_VULKAN:BOOL=ON -DTI_WITH_LLVM:BOOL=OFF -DTI_EXPORT_CORE:BOOL=ON" python setup.py develop

The libtaichi_c_api.so can be found at taichi-repo/_skbuild/linux-x86_64-3.9/cmake-install/c_api/lib and the libtaichi_export_core.so can be found at taichi-repo/_skbuild/linux-x86_64-3.9/cmake-install/lib.

[PENGUINLIONG]

Since it's now a part of the wheel, I'm closing this.