mq-container
mq-container copied to clipboard
ibm-messaging
mqweb does not stop / restart
I'm just running the latest container from dockerhub. It starts fine but if I stop and restart the mqweb component does not work any longer. Same behavior with and without volumes attached to the container. 9.2.0.0 works fine
Regards Joerg
2021-03-22T15:33:06.951Z Error 1 starting web server: Server mqweb is already running. 2021-03-22T15:33:06.951Z Error starting web server: exit status 1
Can you clarify the steps you're taking to see the error, please? I've just tried restart the web server, and it seemed to work:
docker run -d -e LICENSE=accept ibmcom/mq
4b5c1547289d532109ec13e058c2bf2f73edd5a3942ee73451d743e5ef20c278
$ docker exec -ti 4b5 bash
bash-4.4$ ps -ef
UID PID PPID C STIME TTY TIME CMD
1001 1 0 2 15:59 ? 00:00:00 runmqserver -nologruntime -dev
1001 52 1 0 15:59 ? 00:00:00 /bin/sh /opt/mqm/bin/strmqweb
1001 152 52 0 15:59 ? 00:00:00 /bin/sh /opt/mqm/web/bin/server start mqweb
1001 175 152 99 15:59 ? 00:00:22 /opt/mqm/java/jre64/jre/bin/java -javaagent:/opt/mqm/web/bin/tools/ws-javaagent.jar -Djava.awt.headless=true -Djdk.attach.allowAttachSelf=true -XX:MaxPermSize=256m -Djdk.tls.ephemeralDHKey
1001 176 152 8 15:59 ? 00:00:00 /opt/mqm/java/jre64/jre/bin/java -javaagent:/opt/mqm/web/bin/tools/ws-javaagent.jar -Djava.awt.headless=true -Djdk.attach.allowAttachSelf=true -jar /opt/mqm/web/bin/tools/ws-server.jar mqw
1001 229 1 1 15:59 ? 00:00:00 /opt/mqm/bin/amqzxma0 -m 4b5c1547289d -x -u mqm
1001 245 229 0 15:59 ? 00:00:00 /opt/mqm/bin/amqzfuma -m 4b5c1547289d
1001 250 229 0 15:59 ? 00:00:00 /opt/mqm/bin/amqzmgr0 -m 4b5c1547289d
1001 253 229 1 15:59 ? 00:00:00 /opt/mqm/bin/amqzmuc0 -m 4b5c1547289d
1001 271 229 0 15:59 ? 00:00:00 /opt/mqm/bin/amqzmur0 -m 4b5c1547289d
1001 300 229 0 15:59 ? 00:00:00 /opt/mqm/bin/amqzmuf0 -m 4b5c1547289d
1001 316 229 0 15:59 ? 00:00:00 /opt/mqm/bin/amqrrmfa -m 4b5c1547289d -t2332800 -s2592000 -p2592000 -g5184000 -c3600
1001 339 250 0 15:59 ? 00:00:00 /opt/mqm/bin/runmqchi -m 4b5c1547289d -q SYSTEM.CHANNEL.INITQ -r
1001 343 250 0 15:59 ? 00:00:00 /opt/mqm/bin/amqpcsea 4b5c1547289d
1001 346 250 0 15:59 ? 00:00:00 /opt/mqm/bin/runmqlsr -r -m 4b5c1547289d -t TCP -p 1414
1001 353 229 0 15:59 ? 00:00:00 /opt/mqm/bin/amqzlaa0 -m4b5c1547289d -fip0
1001 358 300 0 15:59 ? 00:00:00 /opt/mqm/bin/amqfqpub -m4b5c1547289d
1001 391 358 0 15:59 ? 00:00:00 /opt/mqm/bin/amqfcxba -m 4b5c1547289d
1001 446 0 1 15:59 pts/0 00:00:00 bash
1001 454 446 1 15:59 pts/0 00:00:00 ps -ef
bash-4.4$ dspmqweb
MQWB1124I: Server 'mqweb' is running.
MQWB1121E: The status of the mqweb server applications cannot be determined.
A request was made to read the status of the deployed mqweb server applications, however the data was not available. This may indicate that there is already an mqweb server started on this system, probably related to another IBM MQ instance.
Check the startup logs for the mqweb server, looking in particular for conflicting usage of network ports. Ensure that if you have multiple mqweb servers on a system, they are configured to use distinct network ports. Restart the mqweb server and ensure it started correctly. If the problem persists, seek assistance from IBM support.
bash-4.4$ endmqweb
Stopping server mqweb.
Server mqweb stopped.
bash-4.4$ strmqweb
Starting server mqweb.
Server mqweb started with process ID 759.
bash-4.4$ dspmqweb
MQWB1124I: Server 'mqweb' is running.
MQWB1123E: The status of the mqweb server applications cannot be determined.
A request was made to read the status of the deployed mqweb server applications, however the data appears corrupt. This may indicate that there is already an mqweb server started on this system, probably related to another IBM MQ instance.
Check the startup logs for the mqweb server, looking in particular for conflicting usage of network ports. Ensure that if you have multiple mqweb servers on a system, they are configured to use distinct network ports. Restart the mqweb server and ensure it started correctly. If the problem persists, seek assistance from IBM support.
docker run --env LICENSE=accept --env MQ_QMGR_NAME=QM1 --publish 1415:1414 --publish 9445:9443 --detach --volume qm1data:/mnt/mqm ibmcom/mq
docker ps b9a74bfc8be6 ibmcom/mq "runmqdevserver" 51 seconds ago Up 49 seconds 9157/tcp, 0.0.0.0:1415->1414/tcp, 0.0.0.0:9445->9443/tcp unruffled_kare
docker stop b9a74bfc8be6
docker start b9a74bfc8be6
docker logs b9a74bfc8be6 .... 2021-03-22T17:50:42.681Z Error 1 starting web server: Server mqweb is already running. 2021-03-22T17:50:42.681Z Error starting web server: exit status 1
docker exec -ti b9a bash bash-4.4$ dspmqweb MQWB1124I: Server 'mqweb' is running. URLS: https://b9a74bfc8be6:9443/ibmmq/console/ https://b9a74bfc8be6:9443/ibmmq/rest/ bash-4.4$ endmqweb
Stopping server mqweb. The command stop failed because of a communication error with the server. Server mqweb is not running. bash-4.4$ dspmqweb MQWB1124I: Server 'mqweb' is running. URLS: https://b9a74bfc8be6:9443/ibmmq/console/ https://b9a74bfc8be6:9443/ibmmq/rest/ bash-4.4$
Facing the same issue here!
$ docker exec -ti 4a7 bash
bash-4.4$ ps -ef
UID PID PPID C STIME TTY TIME CMD
1001 1 0 0 20:03 ? 00:00:01 runmqserver -nologruntime -dev
1001 102 1 0 20:03 ? 00:00:00 /opt/mqm/bin/amqzxma0 -m test -x -u mqm
1001 117 102 0 20:03 ? 00:00:00 /opt/mqm/bin/amqzfuma -m test
1001 122 102 0 20:03 ? 00:00:00 /opt/mqm/bin/amqzmgr0 -m test
1001 135 102 0 20:03 ? 00:00:00 /opt/mqm/bin/amqzmuc0 -m test
1001 169 102 0 20:03 ? 00:00:00 /opt/mqm/bin/amqzmur0 -m test
1001 183 102 0 20:03 ? 00:00:00 /opt/mqm/bin/amqzmuf0 -m test
1001 203 102 0 20:03 ? 00:00:00 /opt/mqm/bin/amqrrmfa -m test -t2332800 -s2592000 -p2592000 -g5184000 -c3600
1001 240 183 0 20:04 ? 00:00:00 /opt/mqm/bin/amqfqpub -mtest
1001 246 122 0 20:04 ? 00:00:00 /opt/mqm/bin/runmqchi -m test -q SYSTEM.CHANNEL.INITQ -r
1001 249 122 0 20:04 ? 00:00:00 /opt/mqm/bin/amqpcsea test
1001 250 122 0 20:04 ? 00:00:00 /opt/mqm/bin/runmqlsr -r -m test -t TCP -p 1414
1001 253 102 0 20:04 ? 00:00:00 /opt/mqm/bin/amqzlaa0 -mtest -fip0
1001 302 240 0 20:04 ? 00:00:00 /opt/mqm/bin/amqfcxba -m test
1001 362 250 0 20:08 ? 00:00:00 /opt/mqm/bin/amqrmppa -m test
1001 371 0 0 20:10 pts/0 00:00:00 bash
1001 377 371 0 20:10 pts/0 00:00:00 ps -ef
bash-4.4$ strmqweb
Server mqweb is already running.
bash-4.4$ endmqweb
Stopping server mqweb.
The command stop failed because of a communication error with the server.
Server mqweb is not running.
bash-4.4$
I forced a refresh of my docker image (it was a 2 months old). It worked on the first execution but not on subsequent one...
I have the same issue. In a fresh container the web UI works fine, but if the container gets restarted it stops working.
The mqweb logs show some errors related to keystore bindings, I suspect the issue has to do with how mqweb is setting up a cert when it runs for the https connection to the UI, but I have no idea how to fix it.
Yup, I'm seeing the same behavior with 9.2.2.0-r1. When I run my docker run command, the UI comes up correctly. After running docker stop <container_name> followed by a docker start <container_name> the web UI fails to load. For now I've switched back to 9.2.1.0-r1 and the restart works as expected. lmk if there is anything I can provide to help debug.
Run command: docker run --env LICENSE=accept --env MQ_QMGR_NAME=QL20 --publish 1414:1414 --publish 9443:9443 -d ibmcom/mq
For me, I had to downgrade to 9.2.0.0-r1 in order to restart correctly the web console.
Same behavior and same error message running 9.2.3.0-r1. Note that I am on docker for Windows but I assume this should have no influence.
I get the same behavior on latest but also 9.2.1.0-r1. Its log when trying to resume it with docker start
AMQ8024I: IBM MQ channel initiator started. [CommentInsert1(SYSTEM.CHANNEL.INITQ)]
Error 1 starting web server:
Server mqweb is already running.
Error starting web server: exit status 1
AMQ8942I: Starting to process automatic MQSC configuration script.
AMQ8939I: Automatic MQSC configuration script has completed, and contained 22 command(s), of which 0 had errors. [ArithInsert1(22), CommentInsert1(0)]
Also, I noticed if I run the endmqweb and strmqweb while the container is running, even though the mqweb process is running, it no longer is accessible in the web browser.
I am seeing the same issues. The web server starts fine when the container starts, but if stopped and restarted independently it fails with errors connecting to the keystore:
Initial state: ec2-user installations]$ docker exec 45d9db556766 dspmqweb MQWB1124I: Server 'mqweb' is running. URLS: https://45d9db556766:9443/ibmmq/console/ https://45d9db556766:9443/ibmmq/rest/
Stop: [ec2-user@ installations]$ docker exec 45d9db556766 endmqweb
Stopping server mqweb. Server mqweb stopped.
Check: [ec2-user installations]$ docker exec 45d9db556766 dspmqweb MQWB1125I: Server 'mqweb' is not running.
Restart: [ec2-user installations]$ docker exec 45d9db556766 strmqweb
Starting server mqweb. Server mqweb started with process ID 1251.
Check: [ec2-user installations]$ docker exec 45d9db556766 dspmqweb MQWB1124I: Server 'mqweb' is running. MQWB1123E: The status of the mqweb server applications cannot be determined. A request was made to read the status of the deployed mqweb server applications, however the data appears corrupt. This may indicate that there is already an mqweb server started on this system, probably related to another IBM MQ instance. Check the startup logs for the mqweb server, looking in particular for conflicting usage of network ports. Ensure that if you have multiple mqweb servers on a system, they are configured to use distinct network ports. Restart the mqweb server and ensure it started correctly. If the problem persists, seek assistance from IBM support.
The web console is unusable at this point.
Checking the webserver logs gives me the following errors:
Console.log: [WARNING ] CWWKG0033W: The value [${env.AMQ_WEBTRUSTSTOREREF}] specified for the reference attribute [trustStoreRef] was not found in the configuration. [WARNING ] CWPKI0807W: The keyStore location /run/runmqserver/tls/${env.AMQ_WEBKEYSTORE} could not be found for element with id attribute MQWebKeyStore. [ERROR ] CWPKI0033E: The keystore located at /run/runmqserver/tls/${env.AMQ_WEBKEYSTORE} did not load because of the following error: KeyStore "/run/runmqserver/tls/${env.AMQ_WEBKEYSTORE}" does not exist. [WARNING ] CWPKI0809W: There is a failure loading the MQWebKeyStore keystore. If an SSL configuration references the MQWebKeyStore keystore, then the SSL configuration will fail to initialize. [ERROR ] CWPKI0033E: The keystore located at /run/runmqserver/tls/trust.p12 did not load because of the following error: keystore password was incorrect [WARNING ] CWPKI0809W: There is a failure loading the MQWebTrustStore keystore. If an SSL configuration references the MQWebTrustStore keystore, then the SSL configuration will fail to initialize.
Messages.log [8/12/21 8:35:12:453 GMT] 00000023 com.ibm.ws.ssl.config.WSKeyStore I Successfully loaded default keystore: /mnt/mqm/data/web/installations/Installation1/servers/mqweb/resources/security/key.jks of type: JKS [8/12/21 8:35:12:500 GMT] 00000023 com.ibm.ws.ssl.config.WSKeyStore W CWPKI0807W: The keyStore location /run/runmqserver/tls/${env.AMQ_WEBKEYSTORE} could not be found for element with id attribute MQWebKeyStore. [8/12/21 8:35:12:500 GMT] 00000023 com.ibm.ws.ssl.config.WSKeyStore E CWPKI0033E: The keystore located at /run/runmqserver/tls/${env.AMQ_WEBKEYSTORE} did not load because of the following error: KeyStore "/run/runmqserver/tls/${env.AMQ_WEBKEYSTORE}" does not exist. [8/12/21 8:35:12:501 GMT] 00000023 com.ibm.ws.ssl.config.WSKeyStore W CWPKI0809W: There is a failure loading the MQWebKeyStore keystore. If an SSL configuration references the MQWebKeyStore keystore, then the SSL configuration will fail to initialize.
I am seeing the same issues. The web server starts fine when the container starts, but if stopped and restarted independently it fails with errors connecting to the keystore:
Initial state: ec2-user installations]$ docker exec 45d9db556766 dspmqweb MQWB1124I: Server 'mqweb' is running. URLS: https://45d9db556766:9443/ibmmq/console/ https://45d9db556766:9443/ibmmq/rest/
Stop: [ec2-user@ installations]$ docker exec 45d9db556766 endmqweb
Stopping server mqweb. Server mqweb stopped.
Check: [ec2-user installations]$ docker exec 45d9db556766 dspmqweb MQWB1125I: Server 'mqweb' is not running.
Restart: [ec2-user installations]$ docker exec 45d9db556766 strmqweb
Starting server mqweb. Server mqweb started with process ID 1251.
Check: [ec2-user installations]$ docker exec 45d9db556766 dspmqweb MQWB1124I: Server 'mqweb' is running. MQWB1123E: The status of the mqweb server applications cannot be determined. A request was made to read the status of the deployed mqweb server applications, however the data appears corrupt. This may indicate that there is already an mqweb server started on this system, probably related to another IBM MQ instance. Check the startup logs for the mqweb server, looking in particular for conflicting usage of network ports. Ensure that if you have multiple mqweb servers on a system, they are configured to use distinct network ports. Restart the mqweb server and ensure it started correctly. If the problem persists, seek assistance from IBM support.
The web console is unusable at this point.
Checking the webserver logs gives me the following errors:
Console.log: [WARNING ] CWWKG0033W: The value [${env.AMQ_WEBTRUSTSTOREREF}] specified for the reference attribute [trustStoreRef] was not found in the configuration. [WARNING ] CWPKI0807W: The keyStore location /run/runmqserver/tls/${env.AMQ_WEBKEYSTORE} could not be found for element with id attribute MQWebKeyStore. [ERROR ] CWPKI0033E: The keystore located at /run/runmqserver/tls/${env.AMQ_WEBKEYSTORE} did not load because of the following error: KeyStore "/run/runmqserver/tls/${env.AMQ_WEBKEYSTORE}" does not exist. [WARNING ] CWPKI0809W: There is a failure loading the MQWebKeyStore keystore. If an SSL configuration references the MQWebKeyStore keystore, then the SSL configuration will fail to initialize. [ERROR ] CWPKI0033E: The keystore located at /run/runmqserver/tls/trust.p12 did not load because of the following error: keystore password was incorrect [WARNING ] CWPKI0809W: There is a failure loading the MQWebTrustStore keystore. If an SSL configuration references the MQWebTrustStore keystore, then the SSL configuration will fail to initialize.
Messages.log [8/12/21 8:35:12:453 GMT] 00000023 com.ibm.ws.ssl.config.WSKeyStore I Successfully loaded default keystore: /mnt/mqm/data/web/installations/Installation1/servers/mqweb/resources/security/key.jks of type: JKS [8/12/21 8:35:12:500 GMT] 00000023 com.ibm.ws.ssl.config.WSKeyStore W CWPKI0807W: The keyStore location /run/runmqserver/tls/${env.AMQ_WEBKEYSTORE} could not be found for element with id attribute MQWebKeyStore. [8/12/21 8:35:12:500 GMT] 00000023 com.ibm.ws.ssl.config.WSKeyStore E CWPKI0033E: The keystore located at /run/runmqserver/tls/${env.AMQ_WEBKEYSTORE} did not load because of the following error: KeyStore "/run/runmqserver/tls/${env.AMQ_WEBKEYSTORE}" does not exist. [8/12/21 8:35:12:501 GMT] 00000023 com.ibm.ws.ssl.config.WSKeyStore W CWPKI0809W: There is a failure loading the MQWebKeyStore keystore. If an SSL configuration references the MQWebKeyStore keystore, then the SSL configuration will fail to initialize.
UPDATE: Reported this directly to IBM and they have recreated the issue. Working with the platform team to resolve.
Anyone of IBM messaging team has some kind of update about this issue?
The update from the IBM platform team to the ticket I raised above is that this is working as designed now. Control of the web server has been moved to the container level.
Hello I'm using latest image: 9.2.4.0-r1 and I'm having the same problem. If I get latest and run it, it works the first time but if I stop the container and start again I get the above error.
I'm following this tutorial and using the commands provided there: https://developer.ibm.com/tutorials/mq-connect-app-queue-manager-containers/
Workaround is to not use volumes and add the --rm flag to remove the container when it stops:
docker run --rm --env LICENSE=accept --env MQ_QMGR_NAME=QM1 --publish 1414:1414 --publish 9443:9443 --detach --env MQ_APP_PASSWORD=passw0rd ibmcom/mq:latest
I'd also like to mention that I experienced the very same problem using the latest (9.2.4.0-r1) as well as 9.2.1.0-r1.
As per fergardi, mqweb is working fine (post restart) after downgrading to 9.2.0.0-r1.
Hopefully IBM addresses this issue soon.
Thanks all for the inputs, this thread was very helpful!
+1. I have tried icr.io/ibm-messaging/mq:9.2.5.0-r1 and the problem still persists if you run the endmqweb and follow it with strmqweb command:
MQWB1124I: Server 'mqweb' is running.
URLS:
https://853db90b8f32:9443/ibmmq/console/
https://853db90b8f32:9443/ibmmq/rest/
bash-4.4$ endmqweb
Stopping server mqweb.
Server mqweb stopped.
bash-4.4$
bash-4.4$
bash-4.4$ strmqweb
Starting server mqweb.
Server mqweb started with process ID 1085.
bash-4.4$ dspmqweb
MQWB1124I: Server 'mqweb' is running.
MQWB1123E: The status of the mqweb server applications cannot be determined.
A request was made to read the status of the deployed mqweb server applications, however the data appears corrupt. This may indicate that there is already an mqweb server started on this system, probably related to another IBM MQ instance.
Check the startup logs for the mqweb server, looking in particular for conflicting usage of network ports. Ensure that if you have multiple mqweb servers on a system, they are configured to use distinct network ports. Restart the mqweb server and ensure it started correctly. If the problem persists, seek assistance from IBM support.
Another thing, it seems like IBM have deprecated the containers in DockerHub and the new versions of ibm-mq are available in IBM Container Registry.
Me too. Spun up a docker container with the following: docker run --env LICENSE=accept --env MQ_QMGR_NAME=QM1 --publish 1414:1414 --publish 9443:9443 --detach --volume qm1data:/mnt/mqm icr.io/ibm-messaging/mq:latest
IBM MQ queue manager 'QM1' started using V9.3.0.0
Any updates on this?
I was facing the same problem, I was following this tutorial..
I solved the problem deleting the docker volumen where Ibm stores data with the next command:
docker volume rm yourIBMVolume
I created the volumen once again and the container too and problem was fixed !!
I have the same issue with 9.3.1.0 from docker pull icr.io/ibm-messaging/mq:latest Only works the first time, restarts do not work console.log messages.log docker-container.log
There seem to be two sets of people doing different things on this issue:
- Stopping and restarting just the web component within a running container. The code was designed for a container to be an atomic entity, so you restart the whole container, and not just an individual component.
- Stopping and restarting the whole container, and the web component still not working.
I think the original issue was for option (1), which would be for a new feature of the code, which we're not working on, and have no plans for. Therefore I think this issue should be closed.
However, if someone can confirm that option (2) is happening, I think that might be worth a new issue. I cannot reproduce that behaviour with Podman, so the answer may be Docker specific, but we can handle that if someone raising a new issue.
There seem to be two sets of people doing different things on this issue:
- Stopping and restarting just the web component within a running container. The code was designed for a container to be an atomic entity, so you restart the whole container, and not just an individual component.
- Stopping and restarting the whole container, and the web component still not working.
I think the original issue was for option (1), which would be for a new feature of the code, which we're not working on, and have no plans for. Therefore I think this issue should be closed.
However, if someone can confirm that option (2) is happening, I think that might be worth a new issue. I cannot reproduce that behaviour with Podman, so the answer may be Docker specific, but we can handle that if someone raising a new issue.
This issue is about restarting the entire container. Last time I tried it, and as others mentioned above, only deleting the volume solved the issue.
Multiple comments in this issue are specific to just restarting the web component within a running container. The issue has become confused, so I have closed it. If someone wants to raise a new issue for the full container restart, providing full details of how to reproduce, including their environment details, we can try reproduce it and fix the problem. As it is, I cannot reproduce it.
I was able to reproduce the problem, the web console worked the first time when the container was started, I restarted the container and web console stopped working. Error starting web server: exit status 1
I am running Docker Desktop 4.17.0 for windows with WSL2 enabled in docker.
See attached docker log:
docker-container-log.txt