Grpc Load Balancing is not working

[Fordros]

Describe the bug GRPC Load Balancing is not working for several pods in one service. VirtualRouter is sending all requests to only one pod.

Platform EKS Fargate

To Reproduce Create two services (let's call them A and B), with one service (A) receiving traffic from the outside world, and invoking GRPC method on the other service (B). Create multiple pods for service B.

Expected behavior Service (A) make GRPC requests to different instances of service (B). Round-robin strategy...

AppMesh YAML confurigations

apiVersion: appmesh.k8s.aws/v1beta2
kind: VirtualNode
metadata:
  name: service-B-vn
  namespace: app
spec:
  podSelector:
    matchLabels:
      app: service-B
  listeners:
    - portMapping:
        port: 6565
        protocol: grpc
      healthCheck:
        port: 6565
        protocol: grpc
        healthyThreshold: 2
        unhealthyThreshold: 3
        timeoutMillis: 2000
        intervalMillis: 5000
      timeout:
        grpc:
          idle:
            unit: s
            value: 0
          perRequest:
            unit: s
            value: 0
  serviceDiscovery:
    awsCloudMap:
      namespaceName: app
      serviceName: service-B-vs

---
apiVersion: appmesh.k8s.aws/v1beta2
kind: VirtualService
metadata:
  name: service-B-vs
  namespace: app
spec:
  awsName: service-B.app.svc.cluster.local
  provider:
    virtualRouter:
      virtualRouterRef:
        name: service-B-vr

---
apiVersion: appmesh.k8s.aws/v1beta2
kind: VirtualRouter
metadata:
  namespace: app
  name: service-B-vr
spec:
  listeners:
    - portMapping:
        port: 6565
        protocol: grpc
  routes:
    - name: service-B-vr
      grpcRoute:
        match:
          prefix: /
        action:
          weightedTargets:
            - virtualNodeRef:
                name: service-B-vn
              weight: 1`

Environment

• App Mesh controller version: 840364872350.dkr.ecr.us-west-2.amazonaws.com/amazon/appmesh-controller:v1.6.0
• Sidecar Envoy version: 840364872350.dkr.ecr.us-west-2.amazonaws.com/aws-appmesh-envoy:v1.22.2.1-prod
• Kubernetes version: 1.23
• Using EKS Fargate