unexpected requestpath ordering in the ml inference graph meta field

[saeid93]

Describe the bug

I have to produce a simple three-node inference graph with some dummy logic in node just for testing purposes using python SDK and docker wrapper. This is the entire code in each of the containers: node one:

import logging
import os
import numpy as np

class NodeOne(object):
    def predict(self, X, features_names=None):
        return X*2

node two:

import logging
import os
import numpy as np

class NodeTwo(object):
    def predict(self, X, features_names=None):
        return X*3

node three:

import logging
import os
import numpy as np

class NodeThree(object):
    def predict(self, X, features_names=None):
        return X+2

After dockerizing I have deployed the code using separate seldon engine container with following config:

apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
  name: linear-pipeline-separate-pods
spec:
  name: linear-pipeline
  annotations:
    seldon.io/engine-separate-pod: "true"
  predictors:
  - componentSpecs:
    - spec:
        containers:
        - image: sdghafouri/linearmodel:nodeone
          name: node-one
          imagePullPolicy: Always
    - spec:
        containers:
        - image: sdghafouri/linearmodel:nodetwo
          name: node-two
          imagePullPolicy: Always
    - spec:
        containers:
          - image: sdghafouri/linearmodel:nodethree
            name: node-three
            imagePullPolicy: Always
    graph:
      name: node-one
      type: MODEL
      children:
      - name: node-two
        type: MODEL
        children:
        - name: node-three
          type: MODEL
          children: []
    labels:
      sidecar.istio.io/inject: "true"
    name: example

Hitting the endpoint results in the correct answer of ordering 1->2->3, however, the metadata shows another ordering 1->3->2

>>  curl -d '{"data": {"ndarray":[[1.0, 2.0, 5.0, 6.0]]}}'\   
        -X POST http://localhost:32000/seldon/seldon/linear-pipeline-separate-pods/api/v1.0/predictions\
        -H "Content-Type: application/json"
{
  "data": {
    "names": [
      "t:0", 
      "t:1", 
      "t:2", 
      "t:3"
    ], 
    "ndarray": [
      [
        8.0, 
        14.0, 
        32.0, 
        38.0
      ]
    ]
  }, 
  "meta": {
    "requestPath": {
      "node-one": "sdghafouri/linearmodel:nodeone", 
      "node-three": "sdghafouri/linearmodel:nodethree", 
      "node-two": "sdghafouri/linearmodel:nodetwo"
    }
  }
}

I'm not sure if I'm getting right but I think the request path should be same as the nodes computations so if that's correct the meta field should be:

 "meta": {
    "requestPath": {
      "node-one": "sdghafouri/linearmodel:nodeone", 
      "node-two": "sdghafouri/linearmodel:nodetwo",
      "node-three": "sdghafouri/linearmodel:nodethree",
    }

I also tried to

To reproduce

I have the docker images in my public dockerhub so just deploying the aformentioned yaml should give you the same graph. The curl command is also mentioned in the bug description.

Expected behaviour

correct ordering in the meta request path.

Environment

• Cloud Provider: Microk8s
• Kubernetes Cluster Version v1.23.6
• Deployed Seldon System Images: value: docker.io/seldonio/seldon-core-executor:1.13.1 image: docker.io/seldonio/seldon-core-operator:1.13.1

[axsaucedo]

Thank you for reporting thsi @saeid93 - this was added a while back on the metadata routing support, perhaps @RafalSkolasinski may be able to provide relevant insights as well

[RafalSkolasinski]

Hmm... The "requestPath" entry there is a key-value mapping and in Python standard dict the order of items is not guaranteed. I also believe this is the case for json in general

The JSON syntax does not impose any restrictions on the strings used as names, does not require that name strings be unique, and does not assign any significance to the ordering of name/value pairs.

See [1] and [2].

It'd make sense to change it to list of objects, for example:

"requestPath": [
    {"name": "node-one", "image": "sdghafouri/linearmodel:nodeone",}
    {"name": "node-two", "image": "sdghafouri/linearmodel:nodetwo"},
    {"name": "node-three", "image": "sdghafouri/linearmodel:nodethree"},
]

but that would most likely break backwards compatibility for everybody relying on current format.

[ukclivecox]

Please try in v2