Support custom HTTP headers in `HTTPStreamAdapter` and `RestController` configurations

[dominikriemer]

Body

Currently, the HTTP connectors only support very basic user-defined parameters. Users should be able to add custom headers, e.g., for providing authentication options.

StreamPipes Committer

I acknowledge that I am a maintainer/committer of the Apache StreamPipes project.

[CVamsi27]

Hello, I would like to work on this.

[dominikriemer]

Hi @CVamsi27 and welcome! Have you already set up a StreamPipes dev system? I would suggest to start with the sink and continue with the adapter afterwards.

[CVamsi27]

I haven't set up StreamPipes system yet. I will set it up now.

[purplesmurf45]

Hello @dominikriemer, is this issue up for grabs? If so, can you assign it to me?

[tenthe]

Hi @CVamsi27 since you liked the comment of @purplesmurf45 I assume that you are not working on it, right?

@purplesmurf45 thanks a lot, I'll asign you the issue. Let us know if you have any questions.

[purplesmurf45]

Thanks @tenthe @dominikriemer! Regarding the issue, would like me to implement something similar to headers in KafkaController? It would be really helpful if you could elaborate a little on the issue and/or share any resources regarding custom header implementation 🙂

[dominikriemer]

@purplesmurf45 The goal would be to let users provided custom HTTP headers (see a brief discussion at #1450 ). Headers are typically key/value pairs. So we could add an optional configuration option to assign a list of these key/value pairs which would then be used for reading/writing from/to the HTTP endpoint. One example would be Authentication headers (such as Bearer Tokens), provision of content types or any other HTTP header. E.g., in a future extension, we could add a feature to the REST controller to send events in different formats such as JSON or YAML. The Apache HTTP fluent library which is used in the RestPublisher provides several convenience methods to add headers to a request. These would be read from the user-provided configuration. E.g., we could have a CollectionStaticProperty with a group of key/value pairs. Hope this helps!

[debanjanc01]

Hi @dominikriemer , is this still open? I can take this up.

[bossenti]

Yes, it is still open and we would be happy if take this up @debanjanc01 🙂

[debanjanc01]

Please assign this to me. I'll give a go at it @bossenti

[debanjanc01]

Hi @dominikriemer @bossenti While I've gone through the comments on this issue and the discussion on 1450 I'm unable to find the entry points/classes that need change. Are we just talking about the HttpStreamProtocol.java as mentioned in #1450?

Still figuring my way around the codebase. 👨‍💻

[bossenti]

@debanjanc01 yes we are talking about this class

The aim is to extend the configuration of this adapter by a group of additional HTTP headers. This could be solved with a similar UI element as for the S7 adapter

This is just meant to be an inspiration, column names etc. would need to be adjusted 🙂

[dominikriemer]

HI @debanjanc01 so the idea is that users can provide custom key/value pairs for the HTTP headers.

You could think of the following structure for the UI element: A static property collection (which will display a list of the children assigned to the component) with one static property group, which contains two free text properties for the key and the value. The group can be configured to display the children horizontally.

Hope this helps!