kong Plugin precedence of route + service over consumer not working

Is there an existing issue for this?

[x] I have searched the existing issues

Kong version (`$ kong version`)

3.9.1

Current Behavior

In Kubernetes 1.32, when I configure the ratelimit plugin (A) on a KongConsumer, and then configure a different ratelimit plugin (B) on an Ingress and Service, hitting a backend pod which uses this Ingress and Service while authenticated as the KongConsumer shows ratelimit plugin A was used instead of ratelimit plugin B.

Expected Behavior

According to the plugin precedence order, plugins configured on a Route (Ingress) + Service should take precedence over plugins only configured on a Consumer. As such, ratelimit plugin B should have been selected for these requests.

Steps To Reproduce

Create two ratelimit KongClusterPlugins, A and B, which have different ratelimit settings (second,minute, month). Also create an auth KongClusterPlugin for identity, like key-auth.
Create a KongConsumer, Ingress, and Service. The KongConsumer should use plugin A and a configured key-auth plugin for identification, while the Ingress and Service should use plugin B.
Send a request which hits the configured Ingress and Service, using a credential configured in the KongConsumer.

Anything else?

I also posted this in the KIC repo, but am cross-posting it here because this may be more of a gateway issue (I'm not sure).

There are no errors in the KIC with this setup, and the Kong Admin shows the Service, Route, and Consumer are configured as expected.

Here are some example configs that can be used or referenced to reproduce the bug:

apiVersion: configuration.konghq.com/v1
config:
  minute: 900
  month: 15000
  policy: redis
  redis:
    host: my-redis-server
  second: 40
disabled: false
kind: KongClusterPlugin
metadata:
  annotations:
    kubernetes.io/ingress.class: kong
  name: rate-limiting-free
plugin: rate-limiting
---
apiVersion: configuration.konghq.com/v1
config:
  minute: 500
  policy: redis
  redis:
    host: my-redis-server
  second: 40
disabled: false
kind: KongClusterPlugin
metadata:
  annotations:
    kubernetes.io/ingress.class: kong
  name: rate-limiting-bronze
plugin: rate-limiting
---
apiVersion: configuration.konghq.com/v1
config:
  anonymous: anonymous
  hide_credentials: true
  key_names:
  - x-api-key
disabled: false
kind: KongClusterPlugin
metadata:
  annotations:
    kubernetes.io/ingress.class: kong
  labels:
    global: "true"
  name: key-auth-global
plugin: key-auth
---
apiVersion: configuration.konghq.com/v1
credentials:
- key-example-consumer
kind: KongConsumer
metadata:
  annotations:
    konghq.com/plugins: rate-limiting-free
    kubernetes.io/ingress.class: kong
  name: example-consumer
  namespace: my-consumer-namespace
---
apiVersion: v1
data:
  # Randomly generated for this example
  key: ODA3ZmJkNWNjZDA3YmRkYjFmODc1OTE2MDg4NWM5ZGI=
kind: Secret
metadata:
  labels:
    konghq.com/credential: key-auth
  name: key-example-consumer
  namespace: my-consumer-namespace
type: Opaque
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    konghq.com/plugins: rate-limiting-bronze
  name: app
  namespace: my-app-namespace
spec:
  ingressClassName: kong
  rules:
  - host: test.example.com
    http:
      paths:
      - backend:
          service:
            name: app
            port:
              number: 3000
        path: /
        pathType: Prefix
---
apiVersion: v1
kind: Service
metadata:
  annotations:
    konghq.com/plugins: rate-limiting-bronze
  name: app
  namespace: my-app-namespace
spec:
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: http
    port: 3000
    protocol: TCP
    targetPort: http
  selector:
    app.kubernetes.io/name: app
  sessionAffinity: None
  type: ClusterIP

Aug 08 '25 16:08 CharlieC3

Hi @CharlieC3 ,

Sorry, I'm not very familiar with Kubernetes, so I built Kong 3.9.1 locally and tried to reproduce the issue. It seems that, in my case, the priority of Service+Route is indeed higher.

After starting Kong, I did the following:

Created a Service

curl -i -X POST http://localhost:8001/services \
  --data name=test-svc \
  --data url=http://httpbin.org

Created a Route

curl -i -X POST http://localhost:8001/services/test-svc/routes \
  --data name=test-route \
  --data hosts[]=example.com

Created a Consumer

curl -i -X POST http://localhost:8001/consumers \
  --data username=alice

Added a key-auth credential to the Consumer

curl -i -X POST http://localhost:8001/consumers/alice/key-auth \
  --data key=alice-key

Plugin A: bound to the Consumer (limit: 1 request/minute)

curl -i -X POST http://localhost:8001/consumers/alice/plugins \
  --data name=rate-limiting \
  --data config.minute=1 \
  --data config.policy=local

Plugin B: bound to the Route+Service (limit: 2 requests/minute)

curl -i -X POST http://localhost:8001/routes/test-route/plugins \
  --data name=rate-limiting \
  --data config.minute=2 \
  --data config.policy=local

Then I ran:

curl -H "Host: example.com" -H "apikey: alice-key" http://localhost:8000/get

more than three times.

The result was: the first two requests succeeded, and from the third request onwards, they failed — which matches the rate-limiting rule set on Service+Route.

I also tried the kong/kong:3.9.1-ubuntu image from Docker Hub, and got the same result.