Users should be able to configure a bot account and provider service or jwtAuthToken from the UI

[harshach]

Feature If anyone is setting up Ingestion, currently, we ask them to create a service account and go to https://github.com/open-metadata/OpenMetadata/blob/main/conf/openmetadata.yaml#L177 to set up the auth config and restart the server. This is prone to several issues in debugging and getting it right. In some cases, such as google SSO we will ask users to create the private key file and copy that file to the airflow container, which requires volumes to be set up. We are increasing the steps and potential failures when we are asking users to secure the cluster.

Describe the task Improvements

1. We have user-level authentication mechanisms https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-core/src/main/resources/json/schema/entity/teams/user.json#L10
2. UI should allow users to create bot accounts. During this setup, they can select an SSO (service account) or JWT Token
3. If they select an SSO account based on the SSO provider they selected, we should show the required fields for admins to copy token contents
4. Here, it's important that we get an identity created. Example ingestion-bot is pre-added, data quality-bot, etc if the users want to create a different user identity, that's ok as long as they associate the user to the ingest-bot entity. So that in cases of azure where service account identity can be a UUID, they should be able to add that as the user, we will create the respective user and associate with the bot account.
5. In the case JWT token, we need to validate if the JWT token is set up. For docker, we will ship default JWT configs. So all of the docker POC can be done through JWT Token
6. During any workflow deployment, we associate the identity of a bot to a service. Example ingestion bot identity is associated with all the service ingestion workflows, and data quality bot identity is associated with DQ tests deployment and so on..

The above approach allows users to configure everything through the UI and add bot identity and secrets through the UI as well. This will help reduce the no.of steps required to set up.

cc @chirag-madlani @vivekratnavel @mohitdeuex

[harshach]

@nahuelverdugo lets try to close it out in 0.12.1

[nahuelverdugo]

@harshach can we have some UI mockups?

[harshach]

@nahuelverdugo can you take a look

[nahuelverdugo]

Some questions:

• What is the purpose of the Role?
• Do we want to have a fixed botUser, for example, ingestion-bot?
• Once we store the credentials, how do we use them when creating an ingestion workflow from UI and CLI?
• If my OM server has Google SSO enabled, does it make sense to configure, for example, Azure SSO?

[Sachin-chaurasiya]

@nahuelverdugo , Please Let @open-metadata/ui know once backend changes are landed.

[Sachin-chaurasiya]

@devyani-kaushik, we will need an updated mock for this.