custom-op icon indicating copy to clipboard operation
custom-op copied to clipboard

TF 2.5+ GPU build no longer works because CUDA 11 configure is not supported

Open trevor-m opened this issue 4 years ago • 3 comments

With TensorFlow 2.5, there have been some code changes which require CUDA 11. You can read more about the issues when trying to build TF 2.5 against CUDA 10 here: https://github.com/tensorflow/tensorflow/issues/49983 https://github.com/tensorflow/tensorflow/issues/48468

The build rules to configure CUDA in this custom-op repo (https://github.com/tensorflow/custom-op/tree/master/gpu) have not been updated in a long time. Because of that, it is not possible to compile against newer CUDA versions so we can no longer build our custom gpu plugin against newer TF releases.

Questions

  • Will this repository be updated to support newer versions of CUDA? Otherwise, it is impossible to build custom gpu ops with newer TF versions (2.5+).
  • There is an alternative way to build custom ops through using the main TensorFlow repository by placing the custom op code in this folder https://github.com/tensorflow/tensorflow/tree/master/tensorflow/core/user_ops which will reuse the configuration code from the main tensorflow repository. Is this the recommended approach moving forward and will this repository be deprecated? Is this approach equivalent in terms of what kind of plugins can be built?

Thanks!

trevor-m avatar Jul 15 '21 16:07 trevor-m

Hey @trevor-m while this repo appears to be stale, there aren't too many modification that need to be made to support TF2.5+. You can check the changes that addons made: https://github.com/tensorflow/addons/pull/2441/files https://github.com/tensorflow/addons/pull/2218/files

https://github.com/tensorflow/addons/tree/master/build_deps

seanpmorgan avatar Jul 15 '21 16:07 seanpmorgan

Alternatively it does look like you can use that user_ops directory if you're compiling TF from source and not a plugin library

seanpmorgan avatar Jul 15 '21 16:07 seanpmorgan

Hi @seanpmorgan, thanks for the response and for sharing the TF/addons repostiory. Looks like there is also one more PR for cuda 11. https://github.com/tensorflow/addons/pull/1950 Will this repository also be updated with those changes?

Alternatively it does look like you can use that user_ops directory if you're compiling TF from source and not a plugin library

I see, so it is not possible to build a plugin using the user_ops directory?

trevor-m avatar Jul 15 '21 17:07 trevor-m