Tenant CRD: Provision Tenant credentials secret with AccessToken instead of ProviderKey

[eguzki]

Currently when a tenant is created using the Tenant CRD, a secret is created with the credentials of the tenant, namely: a URL and the ProviderKey of the Tenant (or Provider Account).

The 3scale API endpoint to create a new tenant (provider account) returns an access token for the admin user of the newly created tenant. This access token will not be available again. The 3scale API endpoint to get info about a tenant does not return the access token anymore. The operator can use the response of creating a provider account to provision the credentials secret with the access token, instead of the providerKey.

From the reconciliation pattern point of view, this is not a good approach. For instance, what happens if the call to create the secret with the access token fails? The access token cannot be read again so the reconciliation loop cannot fix this issue. On the other hand, the provider key is available and the operator logic is far more resilient to errors and the logic can be run as many times as needed as it can be idempotent. With the access tokens we cannot guarantee idempotency, therefore we cannot follow the reconciliation pattern. The operator should expect issues and reconciling loop to be executed many times, hence, idempotency is a must.

As long as ProviderKeys do not have any limitation in the Account Management API endpoints (with the exception of the Personal Access Token endpoints), I do not see the need to change