bug: meta.filter in plugin doesn't work when we use some variable

[starsz]

Current Behavior

When I use meta.filter in the plugin like this:

curl http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '{
  "plugins": {
    "file-logger": {
         "_meta": {
            "filter": [
                ["upstream_status", "~=", 200]
            ]
        },
      "path": "logs/file.log"
    }
  },
  "upstream": {
    "type": "roundrobin",
    "nodes": {
      "httpbin.org": 1
    }
  },
  "uri": "/*"
}'

What I expect is that when the upstream_status is not equal to 200, then we will write a log into the file. But now, the plugin will be executed even if the upstream_status is 200.

Reason: Because when we do the plugin filter, we can't get the upstream_status (because we hadn't sent a request to upstream), so the value of ngx.var.upstream_status is nil, and the filter is always matched.

https://github.com/apache/apisix/blob/47187faccd96d02773fb1c71980c8ca0ea838d6a/apisix/plugin.lua#L421-L458

Expected Behavior

The meta.filter can work even if the plugin is run at the log phase.

Error Logs

No response

Steps to Reproduce

1. Create route

curl http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '{
  "plugins": {
    "file-logger": {
         "_meta": {
            "filter": [
                ["upstream_status", "~=", 200]
            ]
        },
      "path": "logs/file.log"
    }
  },
  "upstream": {
    "type": "roundrobin",
    "nodes": {
      "httpbin.org": 1
    }
  },
  "uri": "/*"
}'

2. Send two requests

curl localhost:9080/status/200 -v

curl localhost:9080/status/400 -v

3. We can see both requests are logged.

Environment

• APISIX version (run apisix version): 2.15.0
• Operating system (run uname -a):
• OpenResty / Nginx version (run openresty -V or nginx -V):
• etcd version, if relevant (run curl http://127.0.0.1:9090/v1/server_info):
• APISIX Dashboard version, if relevant:
• Plugin runner version, for issues related to plugin runners:
• LuaRocks version, for installation issues (run luarocks --version):

[tokers]

The filter evaluation should be postponed just before the plugin runs.

[soulbird]

Maybe we should run the filter at each phase and cache the results appropriately so the filter can be applied to every plugin.

[tokers]

Maybe we should run the filter at each phase and cache the results appropriately so the filter can be applied to every plugin.

The cache is not necessary but an optional field, should be careful about it.

[tzssangglass]

Perhaps we could add _meta.post_filter for the header_filter, body_filter, and log phases?

[starsz]

Perhaps we could add _meta.post_filter for the header_filter, body_filter, and log phases?

Good idea I think.

[tokers]

Perhaps we could add _meta.post_filter for the header_filter, body_filter, and log phases?

And still reserve the filter? What's the point?

[tzssangglass]

_meta.filter works before request send to upstream, _meta.post_filter works after get response from upstream.

[tokers]

When will the filter run be decided by the plugin itself, isn't it? Two filter There is no sense if we add a post_filter for a plugin that runs before the forwarding.

[tzssangglass]

Two filter There is no sense if we add a post_filter for a plugin that runs before the forwarding.

when we add post_filter, we can do

1. on header_filter, body_filter and log phases, check if plugin config has _meta.post_filter
2. runs _meta.post_filter and _meta.filter, get the plugins that need to be run
3. runs plugins' header_filter, body_filter and log functions

[soulbird]

I think that adding post_filter will increase the difficulty of using APISIX. Users must know at what phase plugin is working, and we do not have any documentation on this aspect. Therefore, before configuring, users also need to read the plugin source code, which I think is a very bad user experience.