Not enough documentation on building the model to serve for What-If Tool

[modbender]

I have seen through both documentation and API documentation for save and many other functions related to model building but I could not find enough documentation about building a model to use as inference in What-If Tool.

At first I did post this on stackoverflow but no results yet: https://stackoverflow.com/questions/73606284/tensorflow-savedmodel-serving-for-what-if-tool-tensorboard

The documentation here: https://www.tensorflow.org/tensorboard/what_if_tool

Says model requirements are: The model(s) you wish to explore must be served using TensorFlow Serving using the classify, regress, or predict API.

I also have downloaded UCI Census Demo Model and have inspected the model signatures and seen serving_default, classification, regression and predict as keys but I have not seen one documentation mention on how to add these input signatures along with it's method name.

I tried this code thinking it might work:

class CustomModel(tf.keras.Module):

  @tf.function(input_signature=[tf.TensorSpec([], tf.int64)])
  def signature_input_fn(self, x):
    if not hasattr(self, 'target'):
        self.target = tf.Variable(1, name=target_column)
    return self.target

model = CustomModel(inputs, result)

tf.saved_model.save(
    model, export_path,
    signatures={
        'serving_default': model.signature_input_fn,
        'classification': model.signature_input_fn,
        'regression': model.signature_input_fn,
        'predict': model.signature_input_fn,
    }
)

But on Tensorboard What-If Tool I got an error saying No classification inputs found in SignatureDef

I don't know much about tensorflow since I'm new to it, all I have is a CSV and a target column. My colleague had written a code to build model using tf.estimatorLinearClassificer for What-If Tool on Jupyter Notebook and it worked without any problem or extra code. But to use it with Tensorboard, the export with serving input receiver function never worked as it kept giving some or other error with model.export_saved_model so I switched to using

https://www.tensorflow.org/tutorials/load_data/pandas_dataframe

to get the inputs and result.

I have been stuck on this problem due to not enough documentation. Due to the nature of my work I can't provide any confidential information and neither the data. Please help.

[singhniraj08]

@modbender,

In Tensorflow What-If Tool documentation, the requirements mentions that The dataset to be inferred by the models must be in a TFRecord file accessible by the TensorBoard web server.

Kindly refer to tutorials for serving a model using TF serving and to convert your data to TFRecord files format.

Hope this helps. Thank you!

[modbender]

I was about to convert the dataset to TFRecord without any problem thanks to this package https://pypi.org/project/pandas-tfrecords/

What I want is converting pandas dataframe with one Target column to saved_model.pb with signatures and signatures method name

Due to no documentation on how to specify method name of a signature my saved model is not being accepted in what if tool

[singhniraj08]

@modbender,

You can save your model using tf.keras.models.save_model function to generate saved_model.pb and keras_metadata.pb files. You can refer to guide for detailed explanation with example to serve your model with tensorflow serving.

For model inferencing on What-If tool, kindly post an issue in tensorboard repo. Thank you!

[modbender]

I think this is something that should be done on this repository as Tensor board documentation on What-If Tool model building leads back to not a documentation but this repository as previously mentioned in the initial comment of this issue

The model(s) you wish to explore must be served using TensorFlow Serving using the classify, regress, or predict API.

[modbender]

To add more about this, the guide link given tells about a SignatureDef with link

We'll use the command line utility saved_model_cli to look at the MetaGraphDefs (the models) and SignatureDefs (the methods you can call) in our SavedModel. See this discussion of the SavedModel CLI in the TensorFlow Guide.

But the link to SignatureDef tutorial returns 404 https://www.tensorflow.org/tfx/tutorials/signature_defs

[singhniraj08]

@modbender,

A PR has been created to fix this. Meanwhile please refer to signature_defs documentation here. Thank you!

[modbender]

@singhniraj08 could I get complete code used for creating UCI Census Demo Model with signatures?

https://storage.googleapis.com/what-if-tool-resources/uci-census-demo/uci-census-demo.zip

[singhniraj08]

@modbender, Please refer here for complete code to setup what-if tool on UCI Census data. For more info, please refer to what-if-tool github repo.

Thank you!

[singhniraj08]

Closing this due to inactivity. Please take a look into the answers provided above, feel free to reopen and post your comments(if you still have queries on this). Thank you!