Is it possible to use the S4TF toolchain with GitHub actions?

[rickwierenga]

Hi, I wondered if it's possible to use the toolchain for GitHub action tests. If no, is there another option for CI tests? Thanks.

[brettkoonce]

you could set up jenkins locally and trigger it on the update, then post back to github.

[dan-zheng]

@rickwierenga: could you please clarify your use case involving GitHub actions and CI?

Currently, Swift for TensorFlow Ubuntu toolchains are published nightly at fixed Google Cloud Storage download links. There's no GitHub hook for when new nightly toolchains are released.

For reference, here's the Dockerfile that tensorflow/swift-apis uses as CI. It clones and installs the latest nightly Ubuntu toolchain and runs swift build/swift test for various repositories.

[rickwierenga]

Thanks. I'd like to run tests for an open source S4TF project I'm working on.

The docker file seems interesting. I'll see if I can use that.

[rickwierenga]

@dan-zheng I checked if I could get that to work, but it crashes with:

CMake Error: The source directory "/swift-apis" does not appear to contain CMakeLists.txt.
277
Specify --help for usage, or press the help button on the CMake GUI.
278
The command '/bin/sh -c cmake -G Ninja -D CMAKE_BUILD_TYPE=Release -D CMAKE_Swift_COMPILER=/swift-tensorflow-toolchain/usr/bin/swiftc -D USE_BUNDLED_CTENSORFLOW=YES -D TensorFlow_INCLUDE_DIR=/swift-tensorflow-toolchain/usr/lib/swift/linux/x86_64/modulemaps/CTensorFlow -D TensorFlow_LIBRARY=/swift-tensorflow-toolchain/usr/lib/swift/linux/libtensorflow.so -B /BinaryCache/tensorflow-swift-apis -S /swift-apis' returned a non-zero code: 1

See https://github.com/rickwierenga/s4tfpackage/runs/432779938 for the full log. It's weird because that repo seems to contain a CMakeLists file: https://github.com/tensorflow/swift-apis/blob/master/CMakeLists.txt. I'm have very little experience with Docker so excuse me if this is a dumb question.

• This is the GitHub Action repo: https://github.com/rickwierenga/S4TF-Tests. I deleted some pieces from the end of the Dockerfile, but that shouldn't be an issue because it crashes before that.
• And this is the sample package I'm trying to use it with https://github.com/rickwierenga/s4tfpackage/.

[brettkoonce]

@rickwierenga i get the same error when trying to run sudo docker build https://raw.githubusercontent.com/tensorflow/swift-apis/master/Dockerfile but if you do a git clone https://github.com/tensorflow/swift-apis.git and then cd swift-apis and sudo docker build . that seems to work.

[rickwierenga]

Strange... The swift-apis repo should not be a dependency for the Dockerfile to work because that's the whole point of having Docker in the first place. Would you happen to know if the Dockerfile could be updated?

[dan-zheng]

@dan-zheng I checked if I could get that to work, but it crashes with:

CMake Error: The source directory "/swift-apis" does not appear to contain CMakeLists.txt.
277
Specify --help for usage, or press the help button on the CMake GUI.
278
The command '/bin/sh -c cmake -G Ninja -D CMAKE_BUILD_TYPE=Release -D CMAKE_Swift_COMPILER=/swift-tensorflow-toolchain/usr/bin/swiftc -D USE_BUNDLED_CTENSORFLOW=YES -D TensorFlow_INCLUDE_DIR=/swift-tensorflow-toolchain/usr/lib/swift/linux/x86_64/modulemaps/CTensorFlow -D TensorFlow_LIBRARY=/swift-tensorflow-toolchain/usr/lib/swift/linux/libtensorflow.so -B /BinaryCache/tensorflow-swift-apis -S /swift-apis' returned a non-zero code: 1

Sorry, tensorflow/swift-apis's CI (Dockerfile) breaks occasionally! It should pass most of the time - when it's broken, our top priority is to fix it.

The CMake error should be fixed now. It occurred at the time because CMake support was just being added and was unstable.

---

My request: could you please clarify what you mean by "how to use the S4TF toolchain with GitHub actions?" The question currently isn't particularly actionable - a specific request would be helpful!

[rickwierenga]

Thanks. I will check it out as soon as I get the chance (probably Sunday) and I'll let you know how it goes.

could you please clarify what you mean by "how to use the S4TF toolchain with GitHub actions?" The question currently isn't particularly actionable - a specific request would be helpful!

William Zhang (@WilliamHYZhang) and I are building an open source project where we would like to run tests online before merging PRs. For most swift packages this is possible through the Swift Action (https://github.com/actions/starter-workflows/blob/master/ci/swift.yml), but it understandably fails for packages requiring third party toolchains. The Dockerfile seems like a great solution for this, if it works :)

[saeta]

@rickwierenga if you get a nice setup, let's figure out how to document it so others can learn from and benefit from your configuration too!

[xanderdunn]

Here is a simple example of what you want to do @rickwierenga: https://github.com/xanderdunn/swift-action

It's much simpler and faster than using the swift-apis Dockerfile, which is getting and building many things. Just grabbing the pre-built binary is sufficient for a GitHub action.