[Plugin]New plugin to invoke a simple REST WebAPI sync

[kumare3]

Motivation: Why do you think this is important?

FlytePlugins provides a simple way to invoke a WebAPI synchronously. Adding a simple plugin that can invoke a REST WebAPI (without auth) may open up a lot of new use-cases. Also the API should have a corresponding Python API of the form

@dataclass
class Outputs():
   ...


t1 = SyncRestAPI(
            url="https://mydomain/xyz?params={{.inputs.x}}&params={{.inputs.y}}",
            inputs=kwargs(x=int,y=int),
            outputs=Outputs,
            timeout=30s,
            retries=3,
           caching....,
)

@workflow
def foo():
    t1(x=...)

the backend plugin can be a simple HTTP Rest call invoker, that passes a json input and receives a json input. this plugin can implement the following API https://github.com/flyteorg/flyteplugins/blob/master/go/tasks/pluginmachinery/webapi/plugin.go#L148-L154 It is important to handle timeouts, retries, http error codes etc.

Outputs can be either a complex JSON or maybe a single item or map etc

Goal: What should the final outcome look like, ideally?

Explained above

Describe alternatives you've considered

using python to invoke and write an API called. This is very expensive owing to the fact of starting a pod.

Propose: Link/Inline OR Additional context

No response

Are you sure this issue hasn't been raised already?

• [X] Yes

Have you read the Code of Conduct?

• [X] Yes

[kumare3]

Further improving on the API calling spec

property	type	description
uri	String	URI for the service. (we could eventually support service discovery)
method	String	HTTP method. One of the GET, PUT, POST, DELETE, OPTIONS, HEAD
accept	String	Accept header as required by server. Defaults to application/json
contentType	String	Content Type - supported types are text/plain, text/html, and application/json (Default)
headers	Map[String, Any]	A map of additional http headers to be sent along with the request.
connectionTimeOut	Integer	Connection Time Out in milliseconds. If set to 0, equivalent to infinity. Default: 100.
readTimeOut	Integer	Read Time Out in milliseconds. If set to 0, equivalent to infinity. Default: 150.

Also add oauth2.0 related client secrets? maybe just the client id? and use the secrets structure to fetch the secrets

We can also support body as a possible input arg, if the method type is post. This body should be a dataclass / struct only? so a forced parmeter called body.

[kumare3]

Eventually we should also support async calls and like a lua script that allows parsing response for completion?

[github-actions[bot]]

Hello 👋, This issue has been inactive for over 9 months. To help maintain a clean and focused backlog, we'll be marking this issue as stale and will close the issue if we detect no activity in the next 7 days. Thank you for your contribution and understanding! 🙏

[hamersaw]

Commenting to keep open.