Self-contained JAR with OS-Arch specific native libraries

[vkuzmin-uber]

Summary:

Improve current fat jar to make it completely self-contained JAR with OS-Arch specific native libraries. New "fat" JAR has embedded resources with JNI library, Neuropod Core and Backend packages at com/uber/neuropod/native/OS-ARCH Issue #457

Test Plan:

Tensorflow and Torchscript Java tests depends on this self-contained JAR.

[codecov[bot]]

Codecov Report

Merging #458 (1db80c6) into master (97d60d6) will increase coverage by 0.05%. The diff coverage is 94.44%.

@@            Coverage Diff             @@
##           master     #458      +/-   ##
==========================================
+ Coverage   88.23%   88.29%   +0.05%     
==========================================
  Files         106      106              
  Lines        6852     6861       +9     
==========================================
+ Hits         6046     6058      +12     
+ Misses        806      803       -3     

Impacted Files	Coverage Δ
...src/main/java/com/uber/neuropod/LibraryLoader.java	86.56% <94.44%> (+7.25%)	:arrow_up:

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 97d60d6...1db80c6. Read the comment docs.

[vkuzmin-uber]

I see Bazel Buildifier Lint failed. Found that there is a tool with linter and formatter of bazel BUILD files. https://github.com/bazelbuild/buildtools/tree/master/buildifier . Fixed.

[VivekPanyam]

I took a look, but I want to spend some time thinking about whether we can come up with a better approach if that's okay.

The point of the recent backend loading refactor was to avoid having to ship backends in a package for each language Neuropod was used from (e.g. pip packages, fat jars, etc).

The approach in this PR has some drawbacks including being limited to one version of TF and one version of Torch. Also, it adds "magic" that many users might not need.

I'm going to comment some alternatives and more general thoughts on the issue you created for this PR (#457)

[vkuzmin-uber]

See #457 (comment) for more details

TL;DR: I think bundling backends into the final JAR and untarring at runtime is specific to your usecase and not very useful outside the constraints of the environment you're running in. It seems like we can solve that problem in a better way by doing it in an internal "installer" library outside of Neuropod.

I removed "pack backend tar" into Fat Jar part and left only Core native bins in specific directories (for OSX and Linux). Please review.

[VivekPanyam]

@vkuzmin-uber Any updates here? Thanks :)