Preserving the host header / DNS record when calling Gateway-ServiceMirror in Multucluster setup

[manju-rn]

What is the issue?

Setup:
1, Multicluster setup - east and west cluster with common trust certs 2. east cluster has a Minio server setup which hosts a service minio and is injected to make it available in west cluster as minio-east 3. A minio client (also meshed) in west cluster attempts to call the minio-east. The linkerd connection itself is successful. However, minio needs the "exact" host name minio and not minio-east. 4. Due to the host mismatch, minio throws error as signature not correct as it geenrates signature using the host name and access key How to retain the "original" host header or DNS entry when calling from "service-mirror" in another cluster?

How can it be reproduced?

1. Setup minio operator in east cluster https://min.io/docs/minio/kubernetes/upstream/operations/installation.html
2. In east cluster, using operator create a minio tenant. This is just a minio server which has a Stateful set and Service minio
3. in east clusrter - inject the minio service with mirror annotation. This would be reflected in the same namespace in west cluster as minio-east
4. In west clustrer, deploy minio client

kind: Deployment
apiVersion: apps/v1
metadata:
  name: minio-client-app
  namespace: minio-ns
  labels:
    app: minio-client
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: minio-client
  template:
    metadata:
      labels:
        app: minio-client
    spec:
      containers:
      - name: minio-client
        image: minio/mc:RELEASE.2024-01-18T07-03-39Z
        imagePullPolicy: IfNotPresent
        stdin: true
        tty: true

exec into the container, and use the MC client of minio to call minio-east service. https://min.io/docs/minio/linux/reference/minio-mc.html

Logs, error output, etc

signature not correct from minio due to host mismatch

output of linkerd check -o short

All checks were passed

Environment

microk8s in both east and west with common trust anchor setup

Possible solution

Someway to retain the host header in gateway or within linkerd proxy when calling the service / pod

Additional context

No response

Would you like to work on fixing this bug?

None

[manju-rn]

Update: A bit of clarity on the above issue and the solution. The real problem is that the Linkerd gateway is actually changing thee host name for minio-east to mino It should have preserved the original host name.

The solution is by using TrafficSplit from Linkerd SMI. described in detailed here https://github.com/minio/mc/issues/4836#issuecomment-1922884106

[manju-rn]

Re-opening this. Want to check whether: The host name minio-east (the mirrored service of minio from east) that is called from west cluster can retain the host name as minio-east when it is received in the east cluster's pod? Currently linkerd changes the host header to match the original host name i.e minio