plugin/req-resp-modifier in an endpoint is not called if one of the backend CEL evaluations fails

[danielfpferreira]

Environment info:

• KrakenD version: 2.0.5
• System info: docker
• Backend technology: Go

Describe the bug plugin/req-resp-modifier in an endpoint is not called if one of the backend CEL evaluations fails

Your configuration file:

{
  "$schema": "https://www.krakend.io/schema/v3.json",
  "version": 3,
  "name": "conditional test",
  "timeout": "3000s",
  "cache_ttl": "300s",
  "output_encoding": "json",
  "disable_rest": false,
  "plugin": {
    "folder": "./plugins/",
    "pattern": ".so"
  },
  "extra_config": {
    "telemetry/logging": {
      "level": "DEBUG",
      "prefix": "[KRAKEND]",
      "syslog": false,
      "stdout": true
    }
  },
  "endpoints": [
    {
      "endpoint": "/services",
      "input_headers": [
        "X-Group-Id"
      ],
      "method": "GET",
      "output_encoding": "json-collection",
      "extra_config": {
        "plugin/req-resp-modifier": {
          "name": [
            "krakend-response-aggregator"
          ]
        }
      },
      "backend": [
        {
          "group": "app1",
          "url_pattern": "/pattern1",
          "encoding": "safejson",
          "sd": "static",
          "method": "GET",
          "host": [
            "localhost:8090"
          ],
          "extra_config": {
            "validation/cel": [
              {
                "check_expr": "'group1' in req_headers['X-Group-Id']"
              }
            ]
          }
        },
        {
          "group": "app2",
          "url_pattern": "/pattern1",
          "encoding": "safejson",
          "sd": "static",
          "method": "GET",
          "host": [
            "localhost:8090"
          ],
          "extra_config": {
            "validation/cel": [
              {
                "check_expr": "'group2' in req_headers['X-Group-Id']"
              }
            ]
          }
        },
        {
          "group": "app3",
          "url_pattern": "/pattern2",
          "encoding": "safejson",
          "sd": "static",
          "method": "GET",
          "host": [
            "localhost:8090"
          ],
          "extra_config": {
            "validation/cel": [
              {
                "check_expr": "'group1' in req_headers['X-Group-Id']"
              }
            ]
          }
        }
      ]
    }
  ]
}

Configuration check output: Result of krakend check -dtc krakend_conditional.json --lint command

Parsing configuration file: krakend_conditional.json
Global settings
        Name: conditional test
        Port: 8080
        Folder: ./plugins/
        Pattern: .so
1 global component configuration(s):
- telemetry/logging
1 API endpoint(s):
        - GET /services
        Timeout: 50m0s
        1 endpoint component configuration(s):
        - plugin/req-resp-modifier
        Connecting to 3 backend(s):
                [+] GET /pattern1
                Timeout: 50m0s
                Hosts: [http://localhost:8090]
                1 backend component configuration(s):
                - validation/cel

                [+] GET /pattern1
                Timeout: 50m0s
                Hosts: [http://localhost:8090]
                1 backend component configuration(s):
                - validation/cel

                [+] GET /pattern2
                Timeout: 50m0s
                Hosts: [http://localhost:8090]
                1 backend component configuration(s):
                - validation/cel

0 async agent(s):
Syntax OK!

Commands used: How did you start the software?

krakend run -d  -c krakend_conditional.json

Expected behavior The objective is to call multiple backends on a endpoint conditionally and return each backend response as a JSON collection where each element is the backend response.

Each backend is called if an HTTP header values matches a condition.

Each backend has a group defined and I have written a response modifier plugin that applied to the endpoint gets the merged response and then collapses each group response and moves the group key name to the value of "name" into the each backend response and adds the response collection to a "collection" object.

So each backend returns something in the likes of

{
  "groupId": "group1",
  "url": "/pattern1"
}

Merged response would look like

{
  "app1": {
    "groupId": "group1",
    "url": "/pattern1"
  },
  "app3": {
    "groupId": "group1",
    "url": "/pattern2"
  }
}

And modified response by the custom plugin:

{
  "collection": [
    {
      "name": "app1",
      "groupId": "group1",
      "url": "/pattern1"
    },
    {
      "name": "app3",
      "groupId": "group1",
      "url": "/pattern2"
    }
  ]
}

With output_encoding as json-collection we would get:

[
  {
    "name": "app1",
    "groupId": "group1",
    "url": "/pattern1"
  },
  {
    "name": "app3",
    "groupId": "group1",
    "url": "/pattern2"
  }
]

Call to services endpoint should return the expected collection resulting of the custom plugin response modification but instead returns an empty JSON collection

 curl http://localhost:8080/services -H 'x-group-id: group1' | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100     2  100     2    0     0    163      0 --:--:-- --:--:-- --:--:--   181
[]

Setting the endpoint's output_encoding to json shows that the backend responses were merged but the response modifier was not triggered.

{
  "app1": {
    "groupId": "group1",
    "url": "/pattern1"
  },
  "app3": {
    "groupId": "group1",
    "url": "/pattern2"
  }
}

Logs:

The custom-plugin has logging and it should be visible with debug

Parsing configuration file: krakend_conditional.json
[KRAKEND] 2022/08/03 - 15:38:34.300 ▶ DEBUG [SERVICE: Plugin Loader] Starting loading process
krakend-response-aggregator loaded
[KRAKEND] 2022/08/03 - 15:38:34.348 ▶ DEBUG [SERVICE: Executor Plugin] plugin #0 (plugins/krakend-response-aggregator.so): plugin: symbol ClientRegisterer not found in plugin krakend-response-aggregator
[KRAKEND] 2022/08/03 - 15:38:34.348 ▶ DEBUG [SERVICE: Handler Plugin] plugin #0 (plugins/krakend-response-aggregator.so): plugin: symbol HandlerRegisterer not found in plugin krakend-response-aggregator
[KRAKEND] 2022/08/03 - 15:38:34.349 ▶ DEBUG [PLUGIN: krakend-response-aggregator] Logger loaded
[KRAKEND] 2022/08/03 - 15:38:34.349 ▶ INFO [SERVICE: Modifier Plugin] Total plugins loaded: 1
[KRAKEND] 2022/08/03 - 15:38:34.349 ▶ DEBUG [SERVICE: Plugin Loader] Loading process completed
[KRAKEND] 2022/08/03 - 15:38:34.349 ▶ DEBUG [SERVICE: Gin] Debug enabled
[KRAKEND] 2022/08/03 - 15:38:34.349 ▶ INFO Starting the KrakenD instance
[KRAKEND] 2022/08/03 - 15:38:34.349 ▶ DEBUG [ENDPOINT: /services] Building the proxy pipe
[KRAKEND] 2022/08/03 - 15:38:34.349 ▶ DEBUG [BACKEND: /pattern1] Building the backend pipe
[KRAKEND] 2022/08/03 - 15:38:34.349 ▶ DEBUG [BACKEND: /pattern1][CEL] Loading configuration
[KRAKEND] 2022/08/03 - 15:38:34.350 ▶ DEBUG [CEL] Parsing expression: 'group1' in req_headers['X-Group-Id']
[KRAKEND] 2022/08/03 - 15:38:34.360 ▶ DEBUG [BACKEND: /pattern1][CEL] 1 preEvaluator(s) loaded
[KRAKEND] 2022/08/03 - 15:38:34.360 ▶ DEBUG [BACKEND: /pattern1][CEL] 0 postEvaluator(s) loaded
[KRAKEND] 2022/08/03 - 15:38:34.360 ▶ DEBUG [BACKEND: /pattern1] Building the backend pipe
[KRAKEND] 2022/08/03 - 15:38:34.360 ▶ DEBUG [BACKEND: /pattern1][CEL] Loading configuration
[KRAKEND] 2022/08/03 - 15:38:34.360 ▶ DEBUG [CEL] Parsing expression: 'group2' in req_headers['X-Group-Id']
[KRAKEND] 2022/08/03 - 15:38:34.360 ▶ DEBUG [BACKEND: /pattern1][CEL] 1 preEvaluator(s) loaded
[KRAKEND] 2022/08/03 - 15:38:34.360 ▶ DEBUG [BACKEND: /pattern1][CEL] 0 postEvaluator(s) loaded
[KRAKEND] 2022/08/03 - 15:38:34.360 ▶ DEBUG [BACKEND: /pattern2] Building the backend pipe
[KRAKEND] 2022/08/03 - 15:38:34.360 ▶ DEBUG [BACKEND: /pattern2][CEL] Loading configuration
[KRAKEND] 2022/08/03 - 15:38:34.361 ▶ DEBUG [CEL] Parsing expression: 'group1' in req_headers['X-Group-Id']
[KRAKEND] 2022/08/03 - 15:38:34.361 ▶ DEBUG [BACKEND: /pattern2][CEL] 1 preEvaluator(s) loaded
[KRAKEND] 2022/08/03 - 15:38:34.361 ▶ DEBUG [BACKEND: /pattern2][CEL] 0 postEvaluator(s) loaded
[KRAKEND] 2022/08/03 - 15:38:34.361 ▶ DEBUG [ENDPOINT: /services][Merge] Backends: 3, sequential: false, combiner: default
[KRAKEND] 2022/08/03 - 15:38:34.361 ▶ DEBUG [ENDPOINT: /services][Modifier Plugins] Adding 0 request and 1 response modifiers
[KRAKEND] 2022/08/03 - 15:38:34.361 ▶ DEBUG [ENDPOINT: /services] Building the http handler
[KRAKEND] 2022/08/03 - 15:38:34.361 ▶ DEBUG [ENDPOINT: /services][JWTSigner] Signer disabled
[KRAKEND] 2022/08/03 - 15:38:34.361 ▶ INFO [ENDPOINT: /services][JWTValidator] Validator disabled for this endpoint
[KRAKEND] 2022/08/03 - 15:38:34.361 ▶ INFO [SERVICE: Gin] Listening on port: 8080
[KRAKEND] 2022/08/03 - 15:38:39.301 ▶ DEBUG [SERVICE: Telemetry] Registering usage stats for Cluster ID uSR1YoKcD2Ts8buAGDYEE9UgLU1ZN1NSKgJqm4dPzJI=
[KRAKEND] 2022/08/03 - 15:41:58.351 ▶ DEBUG [BACKEND: /pattern2][CEL][pre] Evaluator #0 result: true
[KRAKEND] 2022/08/03 - 15:41:58.351 ▶ INFO [BACKEND: /pattern1][CEL][pre] Evaluator #0 result: false
[KRAKEND] 2022/08/03 - 15:41:58.351 ▶ DEBUG [BACKEND: /pattern1][CEL][pre] Evaluator #0 result: true
[KRAKEND] 2022/08/03 - 15:41:58.358 ▶ ERROR [ENDPOINT: /services] Error #0: request aborted by evaluator #0
[GIN] 2022/08/03 - 15:41:58 | 200 |    7.348922ms |       127.0.0.1 | GET      "/services"

Additional comments:

Changing the CEL expression with group2 to group1 so that all backends pass the CEL evaluation proves that the custom plugin works when called

➜ curl http://localhost:8080/services -H 'x-group-id: group1' | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   160  100   160    0     0  16472      0 --:--:-- --:--:-- --:--:-- 17777
[
  {
    "groupId": "group1",
    "name": "app2",
    "url": "/pattern1"
  },
  {
    "groupId": "group1",
    "name": "app3",
    "url": "/pattern2"
  },
  {
    "groupId": "group1",
    "name": "app1",
    "url": "/pattern1"
  }
]

Logs

...
[KRAKEND] 2022/08/03 - 15:52:31.953 ▶ DEBUG [BACKEND: /pattern2][CEL][pre] Evaluator #0 result: true
[KRAKEND] 2022/08/03 - 15:52:31.953 ▶ DEBUG [BACKEND: /pattern1][CEL][pre] Evaluator #0 result: true
[KRAKEND] 2022/08/03 - 15:52:31.953 ▶ DEBUG [BACKEND: /pattern1][CEL][pre] Evaluator #0 result: true
[KRAKEND] 2022/08/03 - 15:52:31.961 ▶ DEBUG [PLUGIN: krakend-response-aggregator] called
[KRAKEND] 2022/08/03 - 15:52:31.962 ▶ DEBUG [PLUGIN: krakend-response-aggregator] response: {map[app1:map[groupId:group1 name:app1 url:/pattern1] app2:map[groupId:group1 name:app2 url:/pattern1] app3:map[groupId:group1 name:app3 url:/pattern2]] true {map[] 0} <nil>}
[GIN] 2022/08/03 - 15:52:31 | 200 |    8.802293ms |       127.0.0.1 | GET      "/services"

[danielfpferreira]

the bug is upstream opened https://github.com/luraproject/lura/issues/597

[github-actions[bot]]

This issue is marked as stale because it has been open over 90 days with no activity. Remove the stale label or comment or this will be closed in 15 days.

[github-actions[bot]]

This issue was closed because it has been marked as stalled for 15 days with no activity.

[github-actions[bot]]

This issue was marked as resolved a long time ago and now has been automatically locked as there has not been any recent activity after it. You can still open a new issue and reference this link.