Unable to renew access token for API client

[ksj-trackunit]

Describe the bug?

We are seeing the following error after upgrading an API from SDK version 16.0.0 to 19.0.0.

After an hour the SDK fails to obtain a new access token via private key authentication. Instead we get the following exception

com.okta.sdk.impl.oauth2.OAuth2TokenRetrieverException: Exception while trying to get OAuth2 access token for client id 0oa102w6icyjxXN6p358
	at com.okta.sdk.impl.oauth2.AccessTokenRetrieverServiceImpl.getOAuth2AccessToken(AccessTokenRetrieverServiceImpl.java:168) ~[okta-sdk-impl-19.0.0.jar!/:19.0.0]
	at com.okta.sdk.impl.oauth2.OAuth2ClientCredentials.refreshOAuth2AccessToken(OAuth2ClientCredentials.java:64) ~[okta-sdk-impl-19.0.0.jar!/:19.0.0]
	at com.okta.sdk.impl.oauth2.OAuth2ClientCredentials.applyToParams(OAuth2ClientCredentials.java:55) ~[okta-sdk-impl-19.0.0.jar!/:19.0.0]
	at com.okta.sdk.resource.client.ApiClient.updateParamsForAuth(ApiClient.java:1226) ~[okta-sdk-api-19.0.0.jar!/:19.0.0]
	at com.okta.sdk.resource.client.ApiClient.invokeAPI(ApiClient.java:1087) ~[okta-sdk-api-19.0.0.jar!/:19.0.0]
...
<private code ommitted>
...
Caused by: com.okta.sdk.impl.oauth2.OAuth2TokenRetrieverException: Exception while trying to get OAuth2 access token for client id <redacted>
	at com.okta.sdk.impl.oauth2.AccessTokenRetrieverServiceImpl.getOAuth2AccessToken(AccessTokenRetrieverServiceImpl.java:168) ~[okta-sdk-impl-19.0.0.jar!/:19.0.0]
	at com.okta.sdk.impl.oauth2.OAuth2ClientCredentials.refreshOAuth2AccessToken(OAuth2ClientCredentials.java:64) ~[okta-sdk-impl-19.0.0.jar!/:19.0.0]
	at com.okta.sdk.impl.oauth2.OAuth2ClientCredentials.applyToParams(OAuth2ClientCredentials.java:55) ~[okta-sdk-impl-19.0.0.jar!/:19.0.0]
	at com.okta.sdk.resource.client.ApiClient.updateParamsForAuth(ApiClient.java:1226) ~[okta-sdk-api-19.0.0.jar!/:19.0.0]
	at com.okta.sdk.resource.client.ApiClient.invokeAPI(ApiClient.java:1087) ~[okta-sdk-api-19.0.0.jar!/:19.0.0]
	at com.okta.sdk.impl.oauth2.AccessTokenRetrieverServiceImpl.getOAuth2AccessToken(AccessTokenRetrieverServiceImpl.java:138) ~[okta-sdk-impl-19.0.0.jar!/:19.0.0]

What is expected to happen?

We expect the access token to be renewed without exceptions.

What is the actual behavior?

The exception above in the issue description occurs after 1 hour and the application is unable to get a new access token.

Restarting the application solves the issue for an hour.

Reproduction Steps?

Create a Spring Boot application with an ApiClient bean and let is run for an hour after which any call to Okta will fail with the above exception.

Additional Information?

The following code is used to setup the API client bean in Spring

@Bean
public ApiClient oktaClient(OktaConfigurationProperties properties) {
    return Clients.builder()
            .setOrgUrl(properties.getClient().getOrgUrl())
            .setClientId(properties.getClient().getClientId())
            .setAuthorizationMode(AuthorizationMode.PRIVATE_KEY)
            .setPrivateKey(properties.getClient().getPrivateKey())
            .setScopes(Set.of("okta.users.manage", "okta.groups.manage"))
            .build();
}

API beans are created like so

@Bean
public UserApi oktaUserApi(ApiClient oktaClient) {
    return new UserApi(oktaClient);
}

Java Version

openjdk 21 2023-09-19 OpenJDK Runtime Environment (build 21+35-2513) OpenJDK 64-Bit Server VM (build 21+35-2513, mixed mode, sharing)

SDK Version

19.0.0

OS version

No response

[arvindkrishnakumar-okta]

Thanks for posting!

I'll take a look.

[jalbinson]

I have seen the same behavior in 19.0.1. It results in a stack overflow error after getting caught in a loop of attempting to refresh the access token by using the api client.

[ybudweiser]

Any workaround for this?

[jalbinson]

Downgrading to 18.0.0 appears to have resolved the issue for me.

[arvindkrishnakumar-okta]

@ybudweiser an easy hack/workaround is to catch the exception and reinitialize the client.

[arvindkrishnakumar-okta]

@jalbinson I'll try to reproduce the issue in 19.x, are you saying the token refresh api call is being retried several times? are you sure there's no network level issue that prevents the token refresh call to fail and retry?

Are you sure it works with 18.x and NOT with 19.x? Ideally, there is no change to this mechanism introduced in 19.x from 18.x and there should be no difference.

[jalbinson]

My guess is it was this PR here: https://github.com/okta/okta-sdk-java/pull/1555/files

By no longer nulling out the access token, we get stuck in a loop of continually trying to refresh the access token when making a call using the ApiClient.

[arvindkrishnakumar-okta]

Fix merged to main branch, release will be available soon.

[arvindkrishnakumar-okta]

Next version 20.0.0 is released.

[wallind]

lol. I just spent the last two days chasing this, and prepping to submit an Issue to you guys here....guess I was too slow.

I do have some thoughts I'll share here though.

When looking at the problem myself and with my team internally, we were prepared to suggest that the fix ought to happen in AccessTokenRetrieverServiceImpl.getOAuth2AccessToken(..) (line 150 currently on main):

From what I can tell, there is no reason that oauth2 should be passed in to authNames here because the authentication will be placed in form parameters, and so ApiClient shouldn't need to be concerned with applying auth itself. This effectively cuts this recursion off closer to the head of where it starts in my eyes.