No ability in plugin schema to have a record with dynamic fields

[jaxtonw]

Is there an existing issue for this?

• [X] I have searched the existing issues

Kong version ($ kong version)

Kong 2.8.4

Current Behavior

When defining a Kong plugin's schema, there is no way to allow for a record that does not have it's field attribute defined.

Minimal example:

local typedefs = require "kong.db.schema.typedefs"

return {
  name = "example",
  fields = {
    { consumer = typedefs.no_consumer },
    { config = {
        type = "record",
        fields = {
          { some_string = { type = "string", required = true } },
          { dynamic_record = { type = "record", required = false } }
        }
      }
    }
  }
}

The field some_string is fine, however dynamic_record produces a schema violation error: dynamic_record: field of type 'record' must declare 'fields'.

The inability to explicitly allow for a dynamic record as one of the fields is limiting. I want my users to be able to explicitly specify arbitrary key and value pairs as dynamic_record. I want both of these dynamic_records to be valid by the schema.

dynamic_record:
  key1: "value1"

dynamic_record:
  key1: 1
  key2: "other_value"

Expected Behavior

Should accept an arbitrary record type as a field

Steps To Reproduce

1. Create a plugin schema
2. Have one of the fields be of type record and do not define a field

Anything else?

The lack of documentation about fields that are of type record is ultimately the most addressable concern here. However, there should be a way to work around this as well.

[jaxtonw]

Update:

A workaround was attempted that was not successful. Setting the fields for the record as fields = {} resulted in the schema violation error going away. However, when applying a configuration where dynamic_record has an added attribute, the configuration fails, as expected.

[Water-Melon]

No, currently this is not supported. However, there is another workaround, which is to define dynamic_record as a string. Then, in your plugin, try to decode this string as JSON, for example:

local record = cjson.decode(conf.dynamic_record)

[github-actions[bot]]

This issue is marked as stale because it has been open for 14 days with no activity.

[jaxtonw]

thanks for the suggestion @Water-Melon! That was quite helpful. I ended up doing something slightly different though.

local typedefs = require "kong.db.schema.typedefs"

return {
  name = "example",
  fields = {
    { consumer = typedefs.no_consumer },
    { config = {
        type = "record",
        fields = {
          { some_string = { type = "string", required = true } },
          { dynamic_record = { type = "array", required = false,
            elements = {
                type = "record",
                fields = {
                  { key= { type = "string", required = true } },
                  { value= { type = "string", required = true } }
                }
              }
            } 
          }
        }
      }
    }
  }
}

This allowed me to also enforce types for both the keys and values. Not quite what was originally desired, but it works for me.

dynamic_record:
  - key: "keyname"
    value: "value"

Going forward, we should document that dynamic records are not supported and that these are valid workarounds when desired.

[github-actions[bot]]

This issue is marked as stale because it has been open for 14 days with no activity.

[github-actions[bot]]

Dear contributor,

We are automatically closing this issue because it has not seen any activity for three weeks. We're sorry that your issue could not be resolved. If any new information comes up that could help resolving it, please feel free to reopen it.

Your contribution is greatly appreciated!

Please have a look our pledge to the community for more information.

Sincerely, Your Kong Gateway team