DeepSpeech icon indicating copy to clipboard operation
DeepSpeech copied to clipboard

Re-think TensorFlow / DeepSpeech build dance

Open lissyx opened this issue 4 years ago • 5 comments

Now that we masterize better the build-and-cache aspect of TaskCluster, it might be a good time to re-think the interfacing for the known-to-be-painful build dance between TensorFlow and TaskCluster:

  • [x] make TensorFlow a git submodule #3107
  • [x] only build libdeepspeech (tensorflow and tflite) #3107
  • [x] move tensorflow taskcluster bits into deepspeech repo: one place to build them all #3107
  • [x] leverage bazel-level caching for build artifacts #3107
  • [x] rework the current taskcluster build-or-cache templates to be more generic and use them for tensorflow? (or copy/pasta) #3068
  • [x] re-do macOS workers setup #3284 #3289
  • [ ] interface bazel build with make ?

lissyx avatar Apr 08 '20 17:04 lissyx

Can I help on this @lissyx

imskr avatar May 30 '20 17:05 imskr

Can I help on this @lissyx

Well maybe not the TC things, but if you want to experiment smoother flow with tensorflow as a submodule, you're welcome

lissyx avatar May 30 '20 22:05 lissyx

Cool. Which branch we have to make git submodule? @lissyx

imskr avatar May 31 '20 06:05 imskr

So I have a PR with:

  • tensorflow as a submodule
  • taskcluster scripts / definitions from tensorflow moved into deepspeech
  • task graph dependencies between deepspeech and tensorflow
  • definitions of tensorflow builds that should be using the generic build/cache mechanism

still a lot to complete, but it's on track.

lissyx avatar Jun 26 '20 12:06 lissyx

* [ ]  interface `bazel build` with `make` ?

@reuben Maybe GitHub Action move is a good time to address this?

lissyx avatar Mar 25 '21 15:03 lissyx