retina icon indicating copy to clipboard operation
retina copied to clipboard

unable to see the adv_drop_count in the podlevel metrics

Open abhipatwari17 opened this issue 1 year ago • 4 comments

I am using retina in my project . i have enabled the advance level pod metrics. I am able to see the adv_forward_count metrics , drop_bytes, drop_count and so . But i am unable to get the metrics of adv_drop_count . Please let me know if i have to add anything . i have attached MetricsConfiguration file below.

apiVersion: retina.sh/v1alpha1
kind: MetricsConfiguration
metadata:
  name: metricsconfigcrd
spec:
  contextOptions:
    - metricName: drop_count
      sourceLabels:
        - ip
        - podname
        - port
      additionalLabels:
        - direction
        - reason
      destinationLabels:
        - ip
        - podname
        - port
        - workload
    - metricName: forward_count
      sourceLabels:
        - ip
        - podname
        - port
      additionalLabels:
        - direction
      destinationLabels:
        - ip
        - podname
        - port
        - workload
  namespaces:
    include:
      - default
      - kube-system

image

abhipatwari17 avatar May 15 '24 07:05 abhipatwari17

Thanks for raising this issue. On my setup, I also don't see any adv_drop_ metrics. We will need to dig into the code to understand why.

I have retina v0.0.5 with the following ConfigMap:

$ k get cm retina-config -oyaml
apiVersion: v1
data:
  config.yaml: |-
    apiServer:
      host: 0.0.0.0
      port: 10093
    logLevel: info
    enabledPlugin: [dropreason,packetforward,linuxutil,dns,packetparser]
    metricsInterval: 10
    enableTelemetry: false
    enablePodLevel: true
    remoteContext: false
    enableAnnotations: true
kind: ConfigMap
metadata:
  annotations:
    meta.helm.sh/release-name: retina
    meta.helm.sh/release-namespace: kube-system
  creationTimestamp: "2024-04-15T06:53:58Z"
  labels:
    app.kubernetes.io/managed-by: Helm
  name: retina-config
  namespace: kube-system

huntergregory avatar May 16 '24 21:05 huntergregory

@abhipatwari17 Can you clarify the following:

  • what's the retina version you are using?
  • can you confirm a network policy manager is running in the system?
  • can you confirm packets are dropped on any one node?
  • can you share the retina-config?
  • can you share the retina-agent and retina-operator status and logs?

anubhabMajumdar avatar May 16 '24 22:05 anubhabMajumdar

Hi , sorry for the late response .let go one by one .... what's the retina version you are using? = "v0.0.10"

can you confirm packets are dropped on any one node? = i am using one node in k8s cluster, i am able to see the drop count level of cluster, i can see there is variation in the graph [not able to see at the pod level]

can you share the retina-config

retina:
  # Default values for retina.
  # This is a YAML-formatted file.
  # Declare variables to be passed into your templates.

  # Support linux and windows by default.
  os:
    linux: true  # Enable retina for Linux
    windows: true # Enable retina for Windows

  operator:
    enabled: true  # Enable retina operator
    repository: ghcr.io/microsoft/retina/retina-operator  # Retina operator repository
    tag: "v0.0.10"
    installCRDs: false
    enableRetinaEndpoint: true
    capture:
      debug: "true"
      jobNumLimit: 0
    resources:
      limits:
        cpu: 500m
        memory: 128Mi
      requests:
        cpu: 10m
        memory: 128Mi

  image:
    repository: ghcr.io/microsoft/retina/retina-agent  # Retina agent repository
    initRepository: ghcr.io/microsoft/retina/retina-init
    pullPolicy: Always
    tag: "v0.0.10"
    # Take the latest version . Ref:https://github.com/microsoft/retina/releases

  enablePodLevel: true
  remoteContext: true
  enableAnnotations: false
  bypassLookupIPOfInterest: false

  namespace: default  # Namespace for retina deployment

  agent:
    name: retina-agent

  agent_win:
    name: retina-agent-win

  retinaPort: 10093  # Port for retina

  apiServer:
    host: "0.0.0.0"
    port: 10093

  # Supported - debug, info, error, warn, panic, fatal.
  logLevel: info

  enabledPlugin_linux: '["packetparser","dropreason","packetforward","linuxutil","dns"]'
  enabledPlugin_win: '["hnsstats"]'

can you share the retina-agent and retina-operator status and logs?

Below are the logs.txt file for both opertor as well as retina agent . please find the attachment.

retina-operator.txt

retina-agent.txt

abhipatwari17 avatar May 31 '24 12:05 abhipatwari17

There was a report at #80. Are these two related? I'm having the same issue of not seeing adv_drop_count metrics and will investigate it further.

crazyboycjr avatar Jun 20 '24 20:06 crazyboycjr