added methods for support ValueProvider in `MqttIO`

[twosom]

Please add a meaningful description for your change here fixes #19378

This PR contains thease changes

• add ValueProvider methods for MqttIO.ConnectionConfiguration

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• [ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
• [ ] Update CHANGES.md with noteworthy changes.
• [ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

[github-actions[bot]]

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @robertwb for label java. R: @johnjcasey for label io.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

[github-actions[bot]]

Reminder, please take a look at this pr: @robertwb @johnjcasey

[github-actions[bot]]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @Abacn for label java. R: @Abacn for label io.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

[github-actions[bot]]

Reminder, please take a look at this pr: @Abacn @Abacn

[github-actions[bot]]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @robertwb for label java. R: @chamikaramj for label io.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

[github-actions[bot]]

Reminder, please take a look at this pr: @robertwb @chamikaramj

[github-actions[bot]]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @Abacn for label java. R: @ahmedabu98 for label io.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

[Abacn]

Hi, is there a requirement in your use case that needs to serve ValueProvider? In the past ValueProviders were mainly used in Dataflow classic templates, which is superceded by flex templates that no longer need ValueProvider

[twosom]

Hi, is there a requirement in your use case that needs to serve ValueProvider? In the past ValueProviders were mainly used in Dataflow classic templates, which is superceded by flex templates that no longer need ValueProvider

@Abacn Hello, thank you for your comment. We are still using the classic template with ValueProvider and we are planning to move to the flex template in the future. In my opinion, once we use flex templates, we may not need ValueProviders, but it would be nice to at least give users the option to use the existing classic templates.

[robertwb]

Now that we have flex templates, we prefer not propagating the ValueProvider boilerplate elsewhere without good reason, and advise flex templates for any template use that uses non-ValueProvider-enabled features. Is there a reason you cannot use flex templates?

[twosom]

Now that we have flex templates, we prefer not propagating the ValueProvider boilerplate elsewhere without good reason, and advise flex templates for any template use that uses non-ValueProvider-enabled features. Is there a reason you cannot use flex templates?

Flex templates require a Docker image to be built, which is more complex than uploading a JAR file to Google Storage. For our company, it's easier to upload the classic templates with a single Gradle task than to use the artifact registry.

[twosom]

This may be specific to our company. I can't guarantee it's the same for other companies. I agree that we shouldn't propagate ValueProviders without a good reason. I'll close the PR for now, but I still think users should have a choice.

[robertwb]

Now that we have flex templates, we prefer not propagating the ValueProvider boilerplate elsewhere without good reason, and advise flex templates for any template use that uses non-ValueProvider-enabled features. Is there a reason you cannot use flex templates?

Flex templates require a Docker image to be built, which is more complex than uploading a JAR file to Google Storage. For our company, it's easier to upload the classic templates with a single Gradle task than to use the artifact registry.

This is good feedback. My reading of https://cloud.google.com/dataflow/docs/guides/templates/using-flex-templates#build-template is that you don't have to build the docker image yourself, just provide the jar and other metadata (just like classic templates). Hopefully it shouldn't be too hard to package this gcloud command in a gradle task directly.

[twosom]

Now that we have flex templates, we prefer not propagating the ValueProvider boilerplate elsewhere without good reason, and advise flex templates for any template use that uses non-ValueProvider-enabled features. Is there a reason you cannot use flex templates?

Flex templates require a Docker image to be built, which is more complex than uploading a JAR file to Google Storage. For our company, it's easier to upload the classic templates with a single Gradle task than to use the artifact registry.

This is good feedback. My reading of https://cloud.google.com/dataflow/docs/guides/templates/using-flex-templates#build-template is that you don't have to build the docker image yourself, just provide the jar and other metadata (just like classic templates). Hopefully it shouldn't be too hard to package this gcloud command in a gradle task directly.

@robertwb Thanks for sharing this article. It will help my team migrate to Flex Templates.