swift-apis icon indicating copy to clipboard operation
swift-apis copied to clipboard

Move TensorFlow module out of stdlib and inject it into toolchains

Open dan-zheng opened this issue 5 years ago • 3 comments

Currently, tensorflow/swift-apis is cloned as an apple/swift build product and is built as part of the Swift standard library. (relevant CMake support)

However, we would like to move TensorFlow out of the stdlib and inject it into toolchains. This would involve:

  • Building a normal toolchain (sans TensorFlow).
  • Running swift build with the toolchain to build TensorFlow.
  • Copying swift build artifacts into the toolchain.

We've investigated ad-hoc ways of doing the steps above, but it turns out SourceKit LSP has set up infrastructure for doing the same thing.

Todos

We can remove the stdlib CMake support for TensorFlow.

We can copy two files:

  • https://github.com/apple/sourcekit-lsp/blob/master/Utilities/build-script-helper.py: mini build script that installs SourceKit LSP into a toolchain. Belongs in this repository.
  • https://github.com/apple/swift/blob/master/utils/swift_build_support/swift_build_support/products/sourcekitlsp.py: a small hook into the script above. Should edit existing tensorflow.py.

End result

  • TensorFlow will no longer be built as part of the standard library. apple/swift compiler developers will not be able to directly import/test TensorFlow.
  • TensorFlow will be injected into toolchains when running utils/build-toolchain-tensorflow, much like SourceKit LSP.

We could also do these steps for the Python module and the Quote module.

dan-zheng avatar Aug 27 '19 18:08 dan-zheng

Randomly, I feel Swift JIRA would be a better fit for this issue, since no change in this repo is necessary for the intended task to complete.

rxwei avatar Aug 27 '19 18:08 rxwei

since no change in this repo is necessary for the intended task to complete.

Utilities/build-script-helper.py should be added to this repository, following the script-by-the same-name from SourceKit LSP.

dan-zheng avatar Aug 27 '19 18:08 dan-zheng

Ok, thanks for the context!

rxwei avatar Aug 27 '19 18:08 rxwei