[Feature][4.1.0] Operation Level Mediation Policy Support

[malinthaprasan]

Describe your problem(s)

We currently only support adding mediation policies in API level or Global level. This is to enable that to per operation level.

From this we hope to support simple policies like:

• Request path
  • Do a resource rewrite
  • Header/query param manipulation
  • Use a different backend URL
  • Transform (json -> xml, xml -> json etc)
  • Validate content (size, schema etc)
  • Call external webhook (blocking or non-blocking)
  • etc
• Response Path
  • Change status code
  • Header/query param manipulation
  • Transform (json -> xml, xml -> json etc)
  • Validate content (size, schema etc)
  • Call external webhook (blocking or non-blocking)
  • etc

Can support more than one policies at a same time.

We should not allow users to directly add Synapse Configs, instead, we can give them a generic set of functions that we can set via the REST API and internally we can convert that to a Synapse Config. At the same time, it should be extensible to convert to Choreo Connect Router configurations as well. Feature should be written in an extensible way to support new operations in the future. Giving ability for users to write new operations is something we need to discuss.

subtasks

• [x] https://github.com/wso2/product-apim/issues/12268
• [ ] https://github.com/wso2/product-apim/issues/11794
• [x] https://github.com/wso2/product-apim/issues/11784
• [x] https://github.com/wso2/product-apim/issues/12269
• [x] https://github.com/wso2/product-apim/issues/11846
• [x] https://github.com/wso2/product-apim/issues/12270
• [x] https://github.com/wso2/product-apim/issues/11822
• [ ] https://github.com/wso2/product-apim/issues/11823
• [x] https://github.com/wso2/product-apim/issues/12271
• [x] https://github.com/wso2/product-apim/issues/12273
• [x] https://github.com/wso2/product-apim/issues/12274
• [x] https://github.com/wso2/product-apim/issues/12272
• [ ] https://github.com/wso2/product-apim/issues/12275
• [x] https://github.com/wso2/product-apim/issues/12276
• [x] https://github.com/wso2/product-apim/issues/12566

[lakshithagunasekara]

The architecture of this feature has been updated for APIM 4.1.0. Instead of a fixed set of different policies, we changed the architecture that can support different policies dynamically.

Each policy will have main two attributes.

1. Policy Specification - All the meta data regarding the policy and it's attributes. This is independent of the gateway.
2. Policy definition - Implementation of policy logic. This definition depends on the gateway and for synapse gateways, definition is a synapse sequence as a j2 template that has mapped values for the policy attributes.

Eg: Set header policy

Policy Specification

File name : setHeader.yaml

  category: Mediation
  name: addHeader
  version: v1
  displayName: Add Header
  description: With this policy, user can add a new header to the request
  applicableFlows:
   - request
   - response
   - fault
  supportedGateways:
   - HTTP
  supportedApiTypes:
   - REST
  policyAttributes:
   -
    name: headerName
    displayName: Header Name
    description: Name of the header to be added
    validationRegex: ^([a-zA-Z_][a-zA-Z\d_\-\ ]*)$
    type: String
    required: true
   -
    name: headerValue
    displayName: Header Value
    description: Value of the header
    validationRegex: ^([a-zA-Z_][a-zA-Z\d_\-\ ]*)$
    type: String
    required: true

Policy definition

File name: setHeader.j2 <property action="set" name="{{headerName}}" value="{{headerValue}}" scope="transport" />

These two files collectively will define an operation policy. At the resource level, the publisher can apply these policies, and based on the policy specification, he has to add the policy Attributes. Eg: from the setHeader policy, the publisher has to enter the new policy header name and value. This UI will be dynamically generated based on the specification.

In the API.yaml, the applied policy will be recorded under each operation with operation policy name and the input values for each operation policy attribute as below.

type: api
version: v4.1.0
data:
  id: 07276f86-aee3-4fd1-9c16-be5d8bc8cb38
  name: PizzaShackAPI
  description: This is a simple API for Pizza Shack online pizza delivery store.
  .
  .
  operations:
   -
    target: /order
    verb: POST
    authType: Application & Application User
    throttlingPolicy: Unlimited
    scopes: []
    usedProductIds: []
    operationPolicies:
      in:
       -
        policyName: add_Auth_header
        polidyId: 11232s-32dfrer232-2323dfeer
        parameters:
          headerName: Authorization
          headerValue: HeaderValue
       -
        policyName: menu_to_pet_rewrite
        polidyId: 32dfrer232-2323dfeer-1232sere
        parameters:
          resourcePath: /Pets
      out:
       -
        policyName: remove_host_header
        polidyId: erqfafafaf-2323dfeer-1232sere
        parameters:
          headerName: Organization
      fault:
       -
        policyName: remove_host_header
        polidyId: asfasfafaw-2323dfqerqreer-1232sere
        parameters:
          headerName: Organization   

This detail will be passed to the gateway during the revision deployment step along with the policy specification and policy definition. Each gateway will have an artifact generator that will understand the policy definition, parse the mapped policy attributes and generate the gateway artifacts.

For synapse gateways, it will generate three sequence as <API_NAME>_<VERSION>--In/Out/Fault.xml. This sequence file will include all the policies applied for this API separated among the resources by a switch case.

Once a request is received for this resource, the operation policies will execute at the mediation level.

[lakshithagunasekara]

New subtasks

• [x] https://github.com/wso2/product-apim/issues/12268
• [x] https://github.com/wso2/product-apim/issues/11784
• [x] https://github.com/wso2/product-apim/issues/12269
• [x] https://github.com/wso2/product-apim/issues/11846
• [x] https://github.com/wso2/product-apim/issues/12270
• [x] https://github.com/wso2/product-apim/issues/11822
• [ ] https://github.com/wso2/product-apim/issues/11823
• [x] https://github.com/wso2/product-apim/issues/12271
• [x] https://github.com/wso2/product-apim/issues/12273
• [x] https://github.com/wso2/product-apim/issues/12274
• [ ] https://github.com/wso2/product-apim/issues/12272
• [ ] https://github.com/wso2/product-apim/issues/12275
• [ ] https://github.com/wso2/product-apim/issues/12276

[lakshithagunasekara]

Features backend was added via https://github.com/wso2/carbon-apimgt/pull/11040