can't get mycounter and mytimer metrics in prometheus following the example

[yaliqin]

Hi,

I followed the custom metrics example https://docs.seldon.io/projects/seldon-core/en/latest/examples/metrics.html and got "mygauge" metrics scraped in Prometheus.

I can get the value of "mygauge" from Prometheus. But I can't get the value of "mycounter" and "mytimer" metrics value in Prometheus.

Describe the bug

To reproduce

1. The model code

import numpy as np
from seldon_core.user_model import SeldonComponent


class Test(SeldonComponent):

    def __init__(self):
        print("enter model")

    def predict(self, input_data: np.ndarray, names=None):
        return np.array(input_data)

    def metrics(self):
        print("metrics called")
        return [
            # a counter which will increase by the given value
            {"type": "COUNTER", "key": "mycounter", "value": 1},

            # a gauge which will be set to given value
            {"type": "GAUGE", "key": "mygauge", "value": 100},

            # a timer (in msecs) which  will be aggregated into HISTOGRAM
            {"type": "TIMER", "key": "mytimer", "value": 20.2},
        ]

2. build image with seldon-core 1.14.0 to "test-model:metrics"
3. deploy yaml file

apiVersion: machinelearning.seldon.io/v1alpha2
kind: SeldonDeployment
metadata:
  name: test-metrics
  namespace: seldon
spec:
  annotations:
    prometheus.io/scrape: "true"
    prometheus.io/path: "/prometheus"
  name: default
  predictors:
    - componentSpecs:
        - spec:
            containers:
              - image: test-model:metrics
                imagePullPolicy: IfNotPresent
                name: test
                env:
                  - name: SELDON_LOG_LEVEL
                    value: DEBUG
                securityContext:
                  runAsUser: 0000
      graph:
        logger:
          mode: all
        children: [ ]
        endpoint:
          type: REST
        type: MODEL
        name: test
      name: default
      replicas: 1`

Expected behaviour

"mycounter","mygauge" and "mytimer" can be checked on prometheus

Environment

seldon-core 1.14.0