Error in an OSGI bundle when trying to acquire a token, "failed to create an XPathFactory."

[lmame]

Hi :)

We fixed the issue, but who knows, other people might have the same problem, so please find our findings below, it might help other people.
Kudos to my colleague Devraj who found out. He will give in a comment more accurate details, as I'm just a UI developer who just plays with Java.

So long story short, we have a server running with OpenJDK 11 and OSGI.
We have a bundle that uses msl4j to acquire a token, so something like that. the code will not work correctly of course, it was just a test:

    @Action(scope = Scope.PUBLIC)
    public String azureToken(
            @ActionParameter(name = "scope") @NotBlank String scope,
            @ActionParameter(name = "authority") @NotBlank String authority,
            @ActionParameter(name = "clientId") @NotBlank String clientId,
            @ActionParameter(name = "username") @NotBlank String username,
            @ActionParameter(name = "password") @NotBlank String password
    ) throws Exception {
        Set<String> scopeSet = new HashSet<>();

        scopeSet.add(scope);

        PublicClientApplication pca = PublicClientApplication.builder(clientId)
                .authority(authority)
                .build();

        UserNamePasswordParameters parameters =
                UserNamePasswordParameters
                        .builder(scopeSet, username, password.toCharArray())
                        .build();

        // Try to acquire a token via username/password. If successful, you should see
        // the token and account information printed out to console
        CompletableFuture<IAuthenticationResult> call = pca.acquireToken(parameters);
        IAuthenticationResult res = call.join();

        return "ok";
    }

Problem:
At runtime, the code was failing in the call.join() with this error message:

java.lang.RuntimeException: XPathFactory#newInstance() failed to create an XPathFactory for the default object model: http://java.sun.com/jaxp/xpath/dom with the XPathFactoryConfigurationException: javax.xml.xpath.XPathFactoryConfigurationException: No XPathFctory implementation found for the object model: http://java.sun.com/jaxp/xpath/dom

The "funny" part is that on some servers it worked, on some it did not and it took us quite some time to figure it out.

Devraj will give more clear details, but in a nutshell when using CompletableFuture the OpenJDK code will using common pool rather than spawning a new ThreadPerTaskExecutor IF the the number of cpu is > 2. https://github.com/openjdk/jdk11u/blob/73eef16128417f4a489c4dde47383bb4a00f39d4/src/java.base/share/classes/java/util/concurrent/CompletableFuture.java#L429

    private static final boolean USE_COMMON_POOL =
        (ForkJoinPool.getCommonPoolParallelism() > 1);

Later used in:

    private static final Executor ASYNC_POOL = USE_COMMON_POOL ?
        ForkJoinPool.commonPool() : new ThreadPerTaskExecutor();

On some of our QAs environments we did not see the issue because they had only 2 CPUS. Since getCommonPoolParallelism() returns the number of CPU - 1 (something like that), on our QA system we would not see the error because we would use a new ThreadPerTaskExecutor.

In our case, using the common pool triggered the error because the thread context is different. It is bundle in our "default" thread or in a new ThreadPerTaskExecutor() and it is basically the "jvm default context" when using the common pool. So our specific jar files could not be found in the common pool thread context. We could have added some jar files in the JVM library path, but it would not have solved the issue if the jars were inside the OSGI bundle for example.

Workaround:
The workaround Devraj found is to force not using the common pool but an ExecutorService to the pca object, so something like that:

    private static final ExecutorService executorService = Executors.newFixedThreadPool(1);
// (...)
        PublicClientApplication pca = PublicClientApplication.builder(clientId)
                .authority(authority)
                .executorService(executorService)
                .build();

"full" code:

// LMA:: Necessary for the workaround.
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class Azure {
    // LMA:: Necessary for the workaround. You can change the number of threads.
    private static final ExecutorService executorService = Executors.newFixedThreadPool(1);

    @Action(scope = Scope.PUBLIC)
    public String azureToken(
            @ActionParameter(name = "scope") @NotBlank String scope,
            @ActionParameter(name = "authority") @NotBlank String authority,
            @ActionParameter(name = "clientId") @NotBlank String clientId,
            @ActionParameter(name = "username") @NotBlank String username,
            @ActionParameter(name = "password") @NotBlank String password
    ) throws Exception {
        Set<String> scopeSet = new HashSet<>();

        scopeSet.add(scope);

        // LMA:: The workaround is to add an executorService to the pca object.
        // Original code:
        // PublicClientApplication pca = PublicClientApplication.builder(clientId)
        //        .authority(authority)
        //        .build();

        // LMA:: Adding the executorService to the pca object:
        PublicClientApplication pca = PublicClientApplication.builder(clientId)
                .authority(authority)
                .executorService(executorService)
                .build();

        UserNamePasswordParameters parameters =
                UserNamePasswordParameters
                        .builder(scopeSet, username, password.toCharArray())
                        .build();

        // Try to acquire a token via username/password. If successful, you should see
        // the token and account information printed out to console
        CompletableFuture<IAuthenticationResult> call = pca.acquireToken(parameters);
        IAuthenticationResult res = call.join();

        return "ok";
    }
}

Hope this helps someone :)

[devr-b]

Thank you Laurent for mentioning me, though you have introduced me to this library and the current use case.

Laurent has covered the entire picture here. I will add the few remaining details.

The PublicClientApplication.acquireToken(UserNamePasswordParameters) method uses one of the methods below to execute the request

• CompletableFuture.supplyAsync(Supplier<U>)

• CompletableFuture.supplyAsync(Supplier<U>, Executor)

depending on whether the builder of the PublicClientApplication instance was provided with an Executor or not.

The problem with not providing an Executor, as seen in our case, was that CompletableFuture will use ForkJoinPool.commonPool() if the parallelism is greater than 1, which is true when the number of processors in the JVM environment is greater than 2. ForkJoinPool worker threads do not have the same context ClassLoader of the calling thread. So, the context ClassLoader of the worker threads may not have access to the classes provided or accessible from the calling thread context, as it happened in our case.

So, if there is a similar issue when acquiring the token, it would be recommended to provide the PublicClientApplication builder with a custom Executor. ForkJoinPool.commonPool() is great for executing tasks and we have used its advantages in other scenarios, but does not seem to be in this current scenario.

[Avery-Dunn]

Hey @lmame and @devr-b , thanks for all this info! We've had a couple users report other issues when using OSGI, so I'm sure there are other users who will run into this issue.

We'll definitely get this workaround into our documentation/error messages to try to help anyone else who encounters this, and will look into adjusting our default executor setup so hopefully a workaround isn't needed.

[lmame]

Cool thanks @Avery-Dunn ^_^