Matching between output tensor name and prediction

[jun-sun]

I trained a multi-task model that has multiple outputs. running saved_model_cli gives the following output signatures The given SavedModel SignatureDef contains the following output(s):

    outputs['A'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 1)
        name: StatefulPartitionedCall:0
    outputs['B'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 1)
        name: StatefulPartitionedCall:1
    outputs['C'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 1)
        name: StatefulPartitionedCall:2
    outputs['D'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 1)
        name: StatefulPartitionedCall:3
  Method name is: tensorflow/serving/predict

When using tfgo to do inference, I cannot find an easy way to match the prediction back to the output tensor name, i.e. ['A', 'B', 'C', 'D']

res1 := p.Model.Exec([]tf.Output{
		p.Model.Op("StatefulPartitionedCall", 1),
	}, input)

How can I map res1 to the prediction of B?

[galeone]

Hi! Do you mean changing the name in the saved model from StatefulPartitionedCall:1 to B?

[jun-sun]

Hi! Do you mean changing the name in the saved model from StatefulPartitionedCall:1 to B?

Just need to match the 4 scores we get from StatefulPartitionedCall:0~3 to the 4 events [A,B,C,D] without running saved_model_cli

[galeone]

:thinking: it should be this way. Can you show me:

1. In python how do you load the saved model, and feed the model to get the values A,B,C,D (and their values)
2. The same thing, but in Go

Because I expect the behavior to be the same, given the same saved model and the same inputs

[jun-sun]

In python, I simply run model.predict(input) which gives me the 4 task output in a python list [array([[0.1545861]], dtype=float32), array([[0.27176663]], dtype=float32), array([[0.04840082]], dtype=float32), array([[0.16918059]], dtype=float32)]

In go, I run

results0 := p.Model.Exec([]tf.Output{
		p.Model.Op("StatefulPartitionedCall", 0),
	}, input)
	results1 := p.Model.Exec([]tf.Output{
		p.Model.Op("StatefulPartitionedCall", 1),
	}, input)
	results2 := p.Model.Exec([]tf.Output{
		p.Model.Op("StatefulPartitionedCall", 2),
	}, input)
	results3 := p.Model.Exec([]tf.Output{
		p.Model.Op("StatefulPartitionedCall", 3),
	}, input)

The order of the output is different

[[0.1691806]]
[[0.04840079]]
[[0.15458608]]
[[0.27176666]]

When I train the model, I name the tasks in the following order

"TaskTypes": [
            "NAVIGATE_IMP",
            "PURCHASE_NAVIGATE",
            "LIKE_NAVIGATE",
            "ADDTOCART_NAVIGATE"
          ],

But running saved_model_cli show ... suggests the output index does not match how I construct the model. The output names seems sorted by alpha order.

outputs['ADDTOCART_NAVIGATE'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 1)
        name: StatefulPartitionedCall:0
    outputs['LIKE_NAVIGATE'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 1)
        name: StatefulPartitionedCall:1
    outputs['NAVIGATE_IMP'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 1)
        name: StatefulPartitionedCall:2
    outputs['PURCHASE_NAVIGATE'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 1)
        name: StatefulPartitionedCall:3

[galeone]

You should first check in Python - after importing the SavedModel with tf.saved_model.load and not with the Keras equivalent - if the output is the expected one.

We need to test this to understand if the SavedModel used in Python in the same way it's used from Go has the same behavior or not. This can give us an hint on where to look (if it's something in common, than there's some pre/post processing to do. If the behavior is different maybe the C library used in Go has some problem :thinking: )

[jun-sun]

OK, I am running

        imported = tf.saved_model.load(model_path)
        f = imported.signatures["serving_default"]
        print(f)

which gives me

 Args:
...
 Returns:
    {'ADDTOCART_NAVIGATE': <1>, 'LIKE_NAVIGATE': <2>, 'NAVIGATE_IMP': <3>, 'PURCHASE_NAVIGATE': <4>}
      <1>: float32 Tensor, shape=(None, 1)
      <2>: float32 Tensor, shape=(None, 1)
      <3>: float32 Tensor, shape=(None, 1)
      <4>: float32 Tensor, shape=(None, 1)

The output seems returned as a map, keyed with output tensor name. Then it explains why the order does not match.

[galeone]

I guess, thus, the problem is solved. I mean, you just need to correctly map the p.Model.Op("StatefulPartitionedCall", index), with the index you expect.

Closing since this is not a tfgo problem.