Not able to load multiple native Go plugins

[matiasinsaurralde]

Branch/Environment/Version

• Branch/Version: All
• Environment: All

Describe the bug When using the tyk-plugin-compiler to build plugins, all generated shared libraries use the same package path. This causes an error when trying to load more than one plugin, for example, when plugin1 is used in api1 and plugin2is used inapi2`:

level=error msg="Could not load Go-plugin" error="plugin.Open("/tmp/test-plugin/plugin1"): plugin already loaded" mwPath="/tmp/test-plugin/plugin1.so" mwSymbolName=AddFooBarHeader

It's important to note that this error isn't related to duplicate handler/hook names but it's caused by a common package path that's used in all Docker-based builds, even if the host directory is a different one, it's mounted under the same path as a Docker volume on every build.

As a workaround my suggestion is to use a dynamic package path, this is also mentioned in golang discussions.

Reproduction steps Steps to reproduce the behavior:

1. Create a directory for your sample plugin (/tmp/sample-plugin) and a file named plugin.go with the following contents:

package main

import (
  "net/http"
)

// AddFooBarHeader adds custom "Foo: Bar" header to the request
func AddFooBarHeader(rw http.ResponseWriter, r *http.Request) {
  r.Header.Add("Foo", "Bar")
}

func main() {}

2. Build a Go plugin:

$ docker run --rm -v `pwd`:/plugin-source tykio/tyk-plugin-compiler:v2.9.4.2 plugin1.so 

3. Build a second version:

$ docker run --rm -v `pwd`:/plugin-source tykio/tyk-plugin-compiler:v2.9.4.2 plugin2.so 

4. Prepare two API definitions, one will load plugin.so, the other one will try to load plugin2.so:

API 1:

    "custom_middleware": {
        "pre": [],
        "post_key_auth": [],
        "auth_check": {},
        "post": [
            {
                "name": "AddFooBarHeader",
                "path": "/tmp/test-plugin/plugin1.so"
            }
        ],
        "driver": "goplugin"
    },

API 2:

    "custom_middleware": {
        "pre": [],
        "post_key_auth": [],
        "auth_check": {},
        "post": [
            {
                "name": "AddFooBarHeader",
                "path": "/tmp/test-plugin/plugin2.so"
            }
        ],
        "driver": "goplugin"
    },

5. The following error will be triggered:

level=error msg="Could not load Go-plugin" error="plugin.Open("/tmp/test-plugin/plugin1"): plugin already loaded" mwPath="/tmp/test-plugin/plugin1.so" mwSymbolName=AddFooBarHeader

6. After the initialization process, only one of the APIs would be executing the plugin. The plugin will be disabled for one of the APIs because of the above error.
7. For testing I use the following docker run string, where the apps directory contains the two API definitions described above:

docker run --name tyk_gateway -p 8080:8080 -v $(pwd)/tyk.conf:/opt/tyk-gateway/tyk.conf -v $(pwd)/apps:/opt/tyk-gateway/apps -v /tmp/test-plugin:/tmp/test-plugin tykio/tyk-gateway

Expected behavior It should be possible to load multiple Go plugins when using the tyk-plugin-compiler.

[letzya]

@matiasinsaurralde @alephnull Anything to add to our docs?

[matiasinsaurralde]

This is fixed by #3196.

[rnienaber-godaddy]

So in this comment was the line:

For now the only strategy is to use different filenames every time the code changes or perform a full process reload.

It seems to be saying that a different plugin name is sufficient for hot reloads.

I built the plugin with a specific logging message:

docker run --rm -v `pwd`:/plugin-source tykio/tyk-plugin-compiler:v3.2.2 plugin2.so

I then changed the message and ran:

docker run --rm -v `pwd`:/plugin-source tykio/tyk-plugin-compiler:v3.2.2 plugin4.so

Then I...

1. Set the config to use plugin2.so and started tyk
2. Updated the config to plugin4.so
3. executed hot reload 3.1 $ curl localhost:8080/tyk/reload/group --header "x-tyk-authorization: foo"

However, same as the original reporter, I get the following error message:

"Could not load Go-plugin" error="plugin.Open("./plugins/plugin4"): plugin already loaded" mwPath="./plugins/plugin4.so" mwSymbolName=AddFooBarHeader

[patriziobruno]

This is fixed by #3196.

what happened to this fix?