TF Java AARCH64 support

[lanking520]

trafficstars

Hi all,

is there any plan to have aarch64 build for tensorflow Java? There is a growing demands from DJL community looking for TF aarch64 support. So far I can see a few blockers using TF JavaCPP approach:

• ARM64 build requires some special packages and optimization from ARM, where they only support TF pip package. Build from source with all optimization could be extremely hard and time consuming. It will be much easier if we can reuse the pip wheel and build the JavaCPP on top.
• ARM64 requires specialized instance for build and test. From DJL team side, we can try to offer a aarch instance for test and verification if needed.

@saudet any thoughts?

[saudet]

We can't rely on the binaries built for Python because most users don't want a dependency on CPython. Although it's very easy to make JavaCPP link with them, and that's something you could consider doing if this is acceptable for your users, see issue https://github.com/tensorflow/java/issues/226#issuecomment-790329027.

There are builds for the C API only but it doesn't expose all the features we (and you) need, so we can't use those either: https://www.tensorflow.org/install/lang_c

Ideally we would need someone to make builds for the C/C++ API, and then JavaCPP can use those easily, similarly to how we can do it with LibTorch in the case of PyTorch, but that's not the case of almost all other libraries, such as OpenCV, FFmpeg, etc that don't offer binary distributions. One of the goals of the JavaCPP Presets is to provide such builds for the Java platform, but that doesn't work for DJL and TF Java who need/want to control their own build process, so I'm wondering under which conditions DJL would be willing to contribute builds to TF Java. Could you offer actual hardware and send over pull requests that can be reviewed here and TF Java remains in control of how the builds get done? Or do you need to keep control of the whole build pipeline, while still somehow avoiding a fork of TF Java, which I'm not entirely clear on how that would happen?

[karllessard]

It seems that Amazon has already started to contribute building official TF binaries, it would be great if they can also support builds that work for non-python clients.

Now having a CPython dependency in a Java process is certainly a thorn in the side but as building TF on our own is getting more cumbersome, it is an option that we might start considering.

[snadampal]

I was able to compile from sources for aarch64-linux (AWS Graviton). And this PR, which is already merged, enables the mkldnn backend as well. Can aarch64-linux be added to CI now?

[Craigacp]

Our CI is Github actions which don't provide aarch64-linux as a hosted runner, so we can't easily make binaries for it.

[learning-to-play]

Hi @MarkDaoust , Do you know who can help?

[snadampal]

Hi @Craigacp , I have added the build support for linux-arm64 (https://github.com/tensorflow/java/pull/545) similar to the other existing platforms. we are happy to work with you to get the platform into CI as well. please let me know where we can collaborate.