serving icon indicating copy to clipboard operation
serving copied to clipboard

Published 20 hours ago • verified publisher icon tensorflow

Support TensorFlow pluggable devices

Open jianyizh opened this issue 1 year ago • 1 comments

PluggableDevice architecture offers a plugin mechanism for registering devices with TensorFlow without the need to make changes in TensorFlow code. It relies on C APIs to communicate with the TensorFlow binary in a stable manner. https://github.com/tensorflow/serving/issues/2038 also asks for support of pluggable devices.

This PR adds the support for pluggable device. It will try to load all .so files under the given directory from argument --tensorflow_plugins by TF_LoadPluggableDeviceLibrary. With current exported symbol "TF_", the package size is increased 4M and total ratio is increased to 101%. We will need to add alwaylink=1 in TensorFlow kernels_experimental library (https://github.com/tensorflow/tensorflow/pull/60786).

jianyizh avatar May 16 '23 05:05 jianyizh

When can this PR be merged into master? any update?

marvin-Yu avatar Jun 01 '23 08:06 marvin-Yu