Resource Reload Throw Exception

[glassfishrobot]

Container.reload(rc) at ContainerRequestFilter or MessageBodyWriter

19:23:53.964 [Grizzly-worker(10)] WARN o.g.grizzly.http.server.HttpHandler - GRIZZLY0200: Service exception org.glassfish.hk2.api.MultiException: A MultiException has 1 exceptions. They are: 1. java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_1,2,1706339275) has been shut down

at org.jvnet.hk2.internal.FactoryCreator.getFactoryHandle(FactoryCreator.java:80) ~[hk2-locator-2.3.0-b05.jar:na] at org.jvnet.hk2.internal.FactoryCreator.dispose(FactoryCreator.java:110) ~[hk2-locator-2.3.0-b05.jar:na] at org.jvnet.hk2.internal.SystemDescriptor.dispose(SystemDescriptor.java:481) ~[hk2-locator-2.3.0-b05.jar:na] at org.glassfish.jersey.process.internal.RequestScope$Instance.remove(RequestScope.java:512) ~[jersey-common-2.11.jar:na] at org.glassfish.jersey.process.internal.RequestScope$Instance.release(RequestScope.java:529) ~[jersey-common-2.11.jar:na] at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:299) ~[jersey-common-2.11.jar:na] at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:254) ~[jersey-server-2.11.jar:na] at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1028) ~[jersey-server-2.11.jar:na] at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:365) ~[jersey-container-grizzly2-http-2.10.1.jar:na] at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:201) ~[grizzly-http-server-2.3.16.jar:2.3.16] at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:175) ~[grizzly-http-server-2.3.16.jar:2.3.16] at org.glassfish.grizzly.http.server.HttpHandlerChain.doHandle(HttpHandlerChain.java:223) ~[grizzly-http-server-2.3.16.jar:2.3.16] at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:235) ~[grizzly-http-server-2.3.16.jar:2.3.16] at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119) ~[grizzly-framework-2.3.16.jar:2.3.16] at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:284) ~[grizzly-framework-2.3.16.jar:2.3.16] at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:201) ~[grizzly-framework-2.3.16.jar:2.3.16] at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:133) ~[grizzly-framework-2.3.16.jar:2.3.16] at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:112) ~[grizzly-framework-2.3.16.jar:2.3.16] at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77) ~[grizzly-framework-2.3.16.jar:2.3.16] at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:561) ~[grizzly-framework-2.3.16.jar:2.3.16] at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:112) ~[grizzly-framework-2.3.16.jar:2.3.16] at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:117) ~[grizzly-framework-2.3.16.jar:2.3.16] at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:56) ~[grizzly-framework-2.3.16.jar:2.3.16] at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:137) ~[grizzly-framework-2.3.16.jar:2.3.16] at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:565) ~[grizzly-framework-2.3.16.jar:2.3.16] at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:545) ~[grizzly-framework-2.3.16.jar:2.3.16] at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25] Caused by: java.lang.IllegalStateException: ServiceLocatorImpl(__HK2_Generated_1,2,1706339275) has been shut down at org.jvnet.hk2.internal.ServiceLocatorImpl.checkState(ServiceLocatorImpl.java:2182) ~[hk2-locator-2.3.0-b05.jar:na] at org.jvnet.hk2.internal.ServiceLocatorImpl.internalGetServiceHandle(ServiceLocatorImpl.java:580) ~[hk2-locator-2.3.0-b05.jar:na] at org.jvnet.hk2.internal.ServiceLocatorImpl.getServiceHandle(ServiceLocatorImpl.java:573) ~[hk2-locator-2.3.0-b05.jar:na] at org.jvnet.hk2.internal.FactoryCreator.getFactoryHandle(FactoryCreator.java:77) ~[hk2-locator-2.3.0-b05.jar:na] ... 26 common frames omitted

Affected Versions

[2.11]

[glassfishrobot]

Reported by icode

[glassfishrobot]

cowwoc said: The problem has to do with server shutdown, not reload. I am getting this while shutting down the server at the end of a unit test, reload() is never invoked.

We're not going to be able to reproduce/fix this issue without the help from Jersey committers. Please add as much information as possible about the incoming request when this exception occurs.

[glassfishrobot]

cowwoc said: Also, I believe this is a regression in version 2.11. I don't recall ever getting it in 2.09 (I upgraded from 2.09 to 2.11).

[glassfishrobot]

icode said:

resourceConfig.registerInstances(new ContainerRequestFilter() {

@Override
public void filter(ContainerRequestContext containerRequestContext) throws IOException {
    container.reload();
}
            });

[glassfishrobot]

icode said: all test code, pls set logger level to warn. all request get a 500 http status

package org.glassfish.jersey.examples.reload;

import org.glassfish.grizzly.http.server.HttpServer;
import org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpServerFactory;
import org.glassfish.jersey.server.ResourceConfig;
import org.glassfish.jersey.server.spi.Container;
import org.glassfish.jersey.server.spi.ContainerLifecycleListener;

import javax.annotation.Priority;
import javax.ws.rs.container.ContainerRequestContext;
import javax.ws.rs.container.ContainerRequestFilter;
import javax.ws.rs.container.PreMatching;
import java.io.*;
import java.net.URI;
import java.util.TimerTask;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
 * Reload example application.
 *
 * A {@link ContainerLifecycleListener container listener} gets registered
 * with the application. Upon application startup notification, the listener schedules
 * a new {@link TimerTask timer task} to check a text file called {@code resources}
 * every 2 seconds. When the text file change is detected, the application gets reloaded with
 * a new {@link ResourceConfig resource configuration} including all
 * resource classes listed in that file.
 *
 * @author Jakub Podlesak (jakub.podlesak at oracle.com)
 */
public class App {

    private static final Logger LOGGER = Logger.getLogger(App.class.getName());
    private static final URI BASE_URI = URI.create("http://localhost:8080/flights/");
    public static final String ROOT_PATH = "arrivals";

    static Container container;

    public static void main(final String[] args) {
        try {
            LOGGER.info("Resource Config Reload Jersey Example App");

            final ResourceConfig resourceConfig = new ResourceConfig(ArrivalsResource.class);
            resourceConfig.registerInstances(new ContainerLifecycleListener() {
@Override
public void onStartup(final Container container) {
    App.container = container;
}

@Override
public void onReload(final Container container) {
    System.out.println("Application has been reloaded!");
}

@Override
public void onShutdown(final Container container) {
    // ignore
}
            });

            resourceConfig.register(ReloadFilter.class);

            final HttpServer server = GrizzlyHttpServerFactory.createHttpServer(BASE_URI, resourceConfig);

            System.out.println(String.format("Application started.\nTry out %s%s\nHit enter to stop it...", BASE_URI, ROOT_PATH));
            System.in.read();
            server.shutdownNow();
        } catch (final IOException ex) {
            Logger.getLogger(App.class.getName()).log(Level.SEVERE, null, ex);
        }
    }

    @PreMatching
    @Priority(0)
    public static class ReloadFilter implements ContainerRequestFilter {

        @Override
        public void filter(ContainerRequestContext containerRequestContext) throws IOException {
            container.reload();
        }
    }
}

[glassfishrobot]

icode said: jersey 2.9 not have this issuse

[glassfishrobot]

cowwoc said: @icode,

Thank you! So now we've established this is a regression relative to version 2.9 and provided a minimal testcase.

[glassfishrobot]

@AdamLindenthal said: Thanks, guys. Moving to backlog.

[glassfishrobot]

cowwoc said: I am running into #2899 and #2894") very frequently (multiple times per run of unit tests). Can you please increase the priority of these issues? It's very annoying trying to differentiate between stack-traces related to actual test failures and these false positives.

[glassfishrobot]

svavra said: Triaged. Let's check if this problem still remains. In case you feel strongly about the issue, please consider contributing a fix by submitting a Github pull request.

[glassfishrobot]

cowwoc said: Stepan,

The problem is still present in Jersey 2.23. If I could pinpoint the problem, I would contribute a fix.

Jersey committers, please let us know if you are able to reproduce the problem on your end as well.

[glassfishrobot]

This issue was imported from java.net JIRA JERSEY-2627

[JPMoresmau]

I confirm the issue occur in Jersey 2.25 and 2.26. Reloading the application inside a request caused an IllegalStateException. Reloading in a different thread as the reload sample shows can work randomly, if there are no requests happening during the reload. But it may happen that the reload interleaves with a request and then it fails because its says "Request scope has been already shut down."