Integration gateway response postfilter [PlanID: 2023.3/45/1.7]

[petergrlica]

Currently there is only support for incoming requests to integration gateway, there needs to be support for response as well, especially coming from workflows.

There's an existing JSON postfilter that should be extended/replaced with the new Response postfilter.

It need to effectively:

• fetch data either from context or from workflows expressions
• feed expression into the configuration
• output data as needed with various http headers

Frontend tasks:

• [ ] header list on modal to add multiple headers to output
• [ ] input (this will be an input for expr values, similar as to what workflows have)

Backend tasks:

• [x] extend JSON postfilter with new Response filter implementation
• [x] tests
• [ ] documentation on various possible responses wf->apigw

The payload example for postfilter configuration on the frontend:

{
    "headers": {
        "content-type": [
            "text/json"
        ]
    },
    "content": "{\"expr\": \"toJSON(records)\", \"type\": \"String\"}"
}

[petergrlica]

Filter inputs

The idea of response postfilter is to have the ability to respond with the data that is currently in the scope (ie from workflow) and the accompanying headers.

Input field

The input field should an expression input (like on the workflows) with the ability to set the type:

• String
• Any
• Array
• KV
• DateTime
• Float
• Integer
• Reader
• Vars

Headers field

The headers field should be a KVV field with the ability to add multiple values per key.

Example:

"content-type": [
  "application/octet-stream"
]

The headers do not need to be draggable, only the add new and remove buttons should exist.

Sending payload to backend

.Postfilter payload for csv

{
    "header": {
        "content-type": [
            "application/octet-stream"
        ],
        "Content-Disposition": [
            "attachment; filename=foo.txt"
        ],
        "Content-Transfer-Encoding": [
            "binary"
        ]
    },
    "input": {
        "expr": "test",
        "type": "Any"
    }
}

.Example of postfilter request (omitted some headers)

curl 'http://localhost:8084/api/system/apigw/filter/${ROUTE_ID}' \
  -H 'Accept: application/json, text/plain, */*' \
  --data-raw '{"routeID":"312308464082747393","weight":"0","kind":"postfilter","ref":"jsonResponse","enabled":true,"params":{"header":{"content-type":["application/octet-stream"],"Content-Disposition":["attachment; filename=foo.txt"],"Content-Transfer-Encoding":["binary"]},"input":{"expr":"test","type":"Any"}}}' \
  --compressed

Workflow <-> apigw response

Name	Value	Type	Conf	Status	Return value
test	"foo"	String	"input":{"expr":"test","type":"String"}	OK
test	"foo"	Any	"input":{"expr":"test","type":"Any"}	OK
test	"[1,2,3]"	Any	"input":{"expr":"test","type":"Any"}	OK	[1,2,3]
test	"[1,2,3]"	Array	"input":{"expr":"test","type":"Any"}	OK	[{"@value":1,"@type":"Float"},{"@value":2,"@type":"Float"},{"@value":3,"@type":"Float"}]
test	{"foo":"bar","baz": [1,2,3]}	Any	"input":{"expr":"test","type":"Any"}	OK	{"baz":{"@value":[1,2,3],"@type":"Any"},"foo":{"@value":"bar","@type":"String"}}
test	{"foo":"bar","baz": [1,2,3]}	KV	"input":{"expr":"test","type":"KV"}	OK	{"baz":"","foo":"bar"}
test	compose record list	Any	"input":{"expr":"test","type":"Any"}	OK	[{"@value":{"recordID":"312298909827465217","moduleID":"312294200882298881","values":[{"name":"AccountName","value":"Account name"},{"name":"OwnerId","value":"312295603054837761"},{"name":"BillingCity","value":"City"},{"name":"Description","value":"\u003cp\u003e\u003cbr\u003e\u003c/p\u003e"},{"name":"Type","value":"Channel Partner / Reseller"},{"name":"GeneratedBillingAddress","value":"City"},{"name":"GeneratedShippingAddress"}],"namespaceID":"312294200496357377","ownedBy":"312295603054837761","createdAt":"2022-11-25T10:50:56Z","createdBy":"312295603054837761"},"@type":"ComposeRecord"},{"@value":{"recordID":"312298937375653889","moduleID":"312294200882298881","values":[{"name":"AccountName","value":"Second account"},{"name":"OwnerId","value":"312295603054837761"},{"name":"BillingStreet","value":"Street"},{"name":"BillingCity","value":"city"},{"name":"BillingCountry","value":"Afghanistan"},{"name":"Description","value":"\u003cp\u003e\u003cbr\u003e\u003c/p\u003e"},{"name":"Industry","value":"Apparel"},{"name":"Type","value":"Customer - Channel"},{"name":"GeneratedBillingAddress","value":"Street\ncity\nAfghanistan"},{"name":"GeneratedShippingAddress"}],"namespaceID":"312294200496357377","ownedBy":"312295603054837761","createdAt":"2022-11-25T10:51:12Z","createdBy":"312295603054837761"},"@type":"ComposeRecord"}]

[petergrlica]

@Fajfa draft PR: https://github.com/cortezaproject/corteza/pull/738 branch: 2023.3.x-feature-apigw-response-postfilter

[petergrlica]

Changelog Entry

Added

What was added

Ability to serve data as response in an integration gateway route.

Why was it added?

An endpoint response gives greater extensibility to integration gateway as different data can be returned.

[martkaczmarek]

@petergrlica is there any way to use it already in workflows? How should this response be passed to the postfilter?