marathon
marathon copied to clipboard
Marathon on CENTOS 7 Fails to start
I have a new install of marathon/mesos/zookeeper on centos 7. I am using the RPMs (1.9.109). Everything fires up OK, but when I go to the marathon interface, it spins with Loading Applications ... and never finishes. The only thing I can find in the logs is this on the master:
INFO Found no roles suitable for revive repetition. (mesosphere.marathon.core.launchqueue.impl.ReviveOffersStreamLogic$ReviveRepeaterLogic:marathon-akka.actor.default-dispatcher-3) Feb 08 10:35:25 nj-dcos01-cl01 marathon[581]: [2020-02-08 10:35:25,513] INFO Found no roles suitable for revive repetition. (mesosphere.marathon.core.launchqueue.impl.ReviveOffersStreamLogic$ReviveRepeaterLogic:marathon-akka.actor.default-dispatcher-2) Feb 08 10:35:30 nj-dcos01-cl01 marathon[581]: [2020-02-08 10:35:30,513] INFO Found no roles suitable for revive repetition. (mesosphere.marathon.core.launchqueue.impl.ReviveOffersStreamLogic$ReviveRepeaterLogic:marathon-akka.actor.default-dispatcher-7) Feb 08 10:35:32 nj-dcos01-cl01 marathon[581]: [2020-02-08 10:35:32,723] INFO Prompting Mesos for a heartbeat via explicit task reconciliation (mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor$$anon$1:marathon-akka.actor.default-dispatcher-8) Feb 08 10:35:32 nj-dcos01-cl01 marathon[581]: [2020-02-08 10:35:32,726] INFO Received fake heartbeat task-status update (mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor:Thread-172)
Firewall/iptables is off. I can't submit a test job to marathon or create an app. Mesos seems to be working OK. sean
I have the same also. I deleted everthing in /var/lib/mesos and /var/lib/zookeeper and still I have this shit. I reported that mesos-dns is not correctly reporting ip addresses almost a year ago and it is still not fixed. I do not get that people are installing this 1GB of DCOS blob with this buggie coding going on there. As if they are a more reliable partner than redhat.
Which version are you using?
this 1.9.109-1.el7 and 1.9.136-1.el7 the same
@f1-outsourcing, you should not blindly delete files. Also, remember that this is all open source. We are happy to accept bug fixes from you.
@seanfulton, sorry for replying so late. Do you still have this issue? If I understood correctly, Marathon fails to load the old state. Was this after an upgrade?
I am deleting files and remove configuration options to see if that results to something. Whatever I am changing I am only able to get marathon-1.7.216-9e2a9b579 working with mesos-1.10.0-2.0.1 I have looked at the change log but cannot seem to find a change that breaks.
This is what I posted to the marathon-framework mailing list:
All of a sudden I having problems with marathon ui getting stuck at 'loading' and end points like http://m01.local:8081/v2/info are not responding (http://m01.local:8081/ping gives pong). I have now downgraded the test cluster to one node, running only mesos-master and zookeeper and marathon. Cleaning between tests the /var/lib/zookeeper and the /var/lib/mesos directories. I have also removed many of the configuration options I had, like ssl etc.
I am only able to get to run marathon-1.7.216-9e2a9b579. marathon-1.8.222-86475ddac and marathon-1.10.17-c427ce965 are having the above mentioned errors/problem.
I have been comparing the marathon 1.7 and marathon 1.8 logs and this what I have noticed. There are quite a bit of log statements missing between 'All services up and running. (mesosphere.marathon.MarathonApp:main' and 'akka://marathon/deadLetters' in the 1.8 log.
Anyone had something similar?
[@mesos-master]# rpm -qa | grep java python-javapackages-3.4.1-11.el7.noarch tzdata-java-2020a-1.el7.noarch java-1.8.0-openjdk-headless-1.8.0.252.b09-2.el7_8.x86_64 javapackages-tools-3.4.1-11.el7.noarch
[@mesos-master]# uname -a Linux m01.local 3.10.0-1127.10.1.el7.x86_64 #1 SMP Wed Jun 3 14:28:03 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
[@mesos-master]# cat /etc/redhat-release CentOS Linux release 7.8.2003 (Core)
marathon 1.8 (unresponsive)
Jun 7 17:40:59 m01 marathon: [2020-06-07 17:40:59,696] INFO All services up and running. (mesosphere.marathon.MarathonApp:main) Jun 7 17:41:13 m01 marathon: [2020-06-07 17:41:13,833] INFO initiate task reconciliation (mesosphere.marathon.MarathonSchedulerActor:marathon-akka.actor.default-dispatcher-9) Jun 7 17:41:13 m01 marathon: [2020-06-07 17:41:13,854] INFO Requesting task reconciliation with the Mesos master (mesosphere.marathon.SchedulerActions:scheduler-actions-thread-0) Jun 7 17:41:13 m01 mesos-master[11203]: I0607 17:41:13.858621 11227 master.cpp:8846] Performing implicit task state reconciliation for framework f5d67e06-6600-4fb9-94dc-a878be2563be-0000 (marathon) at [email protected]:36941 Jun 7 17:41:13 m01 marathon: [2020-06-07 17:41:13,864] INFO task reconciliation has finished (mesosphere.marathon.MarathonSchedulerActor:marathon-akka.actor.default-dispatcher-4)
Jun 7 17:41:13 m01 marathon: [2020-06-07 17:41:13,879] INFO Message [mesosphere.marathon.MarathonSchedulerActor$TasksReconciled$] from Actor[akka://marathon/user/MarathonScheduler/$a#1746491390] to Actor[akka://marathon/deadLetters] was not delivered. [1] dead letters encountered. If this is not an expected behavior, then [Actor[akka://marathon/deadLetters]] may have terminated unexpectedly, This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. (akka.actor.DeadLetterActorRef:marathon-akka.actor.default-dispatcher-7) Jun 7 17:41:13 m01 marathon: [2020-06-07 17:41:13,910] INFO Prompting Mesos for a heartbeat via explicit task reconciliation (mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor$$anon$1:marathon-akka.actor.default-dispatcher-7) Jun 7 17:41:13 m01 mesos-master[11203]: I0607 17:41:13.914615 11228 master.cpp:8889] Performing explicit task state reconciliation for 1 tasks of framework f5d67e06-6600-4fb9-94dc-a878be2563be-0000 (marathon) at [email protected]:36941 Jun 7 17:41:13 m01 marathon: [2020-06-07 17:41:13,924] INFO Received fake heartbeat task-status update (mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor:Thread-13) Jun 7 17:41:28 m01 marathon: [2020-06-07 17:41:28,939] INFO Prompting Mesos for a heartbeat via explicit task reconciliation (mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor$$anon$1:marathon-akka.actor.default-dispatcher-4) Jun 7 17:41:28 m01 mesos-master[11203]: I0607 17:41:28.946494 11229 master.cpp:8889] Performing explicit task state reconciliation for 1 tasks of framework f5d67e06-6600-4fb9-94dc-a878be2563be-0000 (marathon) at [email protected]:36941 Jun 7 17:41:28 m01 marathon: [2020-06-07 17:41:28,950] INFO Received fake heartbeat task-status update (mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor:Thread-14)
marathon 1.7 (ok)
Jun 7 17:37:02 m01 marathon: [2020-06-07 17:37:02,681] INFO All services up and running. (mesosphere.marathon.MarathonApp:main) Jun 7 17:37:06 m01 marathon: [2020-06-07 17:37:06,222] INFO Received TimedCheck (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-8) Jun 7 17:37:06 m01 marathon: [2020-06-07 17:37:06,228] INFO => revive offers NOW, canceling any scheduled revives (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-8) Jun 7 17:37:06 m01 mesos-master[10661]: I0607 17:37:06.232568 10690 master.cpp:5521] Processing REVIVE call for framework f2318310-8c7b-438c-9a9d-48fdf1cd0406-0000 (marathon) at [email protected]:40447 Jun 7 17:37:06 m01 mesos-master[10661]: I0607 17:37:06.232730 10690 hierarchical.cpp:1788] Unsuppressed offers and cleared filters for roles { * } of framework f2318310-8c7b-438c-9a9d-48fdf1cd0406-0000 Jun 7 17:37:06 m01 marathon: [2020-06-07 17:37:06,235] INFO 2 further revives still needed. Repeating reviveOffers according to --revive_offers_repetitions 3 (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-8) Jun 7 17:37:06 m01 marathon: [2020-06-07 17:37:06,238] INFO => Schedule next revive at 2020-06-07T15:37:11.228Z in 4990 milliseconds, adhering to --min_revive_offers_interval 5000 (ms) (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-8) Jun 7 17:37:11 m01 marathon: [2020-06-07 17:37:11,240] INFO Received TimedCheck (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-5) Jun 7 17:37:11 m01 mesos-master[10661]: I0607 17:37:11.246363 10685 master.cpp:5521] Processing REVIVE call for framework f2318310-8c7b-438c-9a9d-48fdf1cd0406-0000 (marathon) at [email protected]:40447 Jun 7 17:37:11 m01 mesos-master[10661]: I0607 17:37:11.246500 10685 hierarchical.cpp:1788] Unsuppressed offers and cleared filters for roles { * } of framework f2318310-8c7b-438c-9a9d-48fdf1cd0406-0000 Jun 7 17:37:11 m01 marathon: [2020-06-07 17:37:11,240] INFO => revive offers NOW, canceling any scheduled revives (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-5) Jun 7 17:37:11 m01 marathon: [2020-06-07 17:37:11,241] INFO 1 further revives still needed. Repeating reviveOffers according to --revive_offers_repetitions 3 (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-5) Jun 7 17:37:11 m01 marathon: [2020-06-07 17:37:11,241] INFO => Schedule next revive at 2020-06-07T15:37:16.240Z in 4999 milliseconds, adhering to --min_revive_offers_interval 5000 (ms) (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-5) Jun 7 17:37:16 m01 marathon: [2020-06-07 17:37:16,261] INFO Received TimedCheck (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-8) Jun 7 17:37:16 m01 mesos-master[10661]: I0607 17:37:16.265516 10689 master.cpp:5521] Processing REVIVE call for framework f2318310-8c7b-438c-9a9d-48fdf1cd0406-0000 (marathon) at [email protected]:40447 Jun 7 17:37:16 m01 mesos-master[10661]: I0607 17:37:16.265655 10689 hierarchical.cpp:1788] Unsuppressed offers and cleared filters for roles { * } of framework f2318310-8c7b-438c-9a9d-48fdf1cd0406-0000 Jun 7 17:37:16 m01 marathon: [2020-06-07 17:37:16,261] INFO => revive offers NOW, canceling any scheduled revives (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-8) Jun 7 17:37:16 m01 marathon: [2020-06-07 17:37:16,409] INFO initiate task reconciliation (mesosphere.marathon.MarathonSchedulerActor:marathon-akka.actor.default-dispatcher-5) Jun 7 17:37:16 m01 marathon: [2020-06-07 17:37:16,437] INFO Requesting task reconciliation with the Mesos master (mesosphere.marathon.SchedulerActions:scheduler-actions-thread-0) Jun 7 17:37:16 m01 mesos-master[10661]: I0607 17:37:16.441344 10686 master.cpp:8846] Performing implicit task state reconciliation for framework f2318310-8c7b-438c-9a9d-48fdf1cd0406-0000 (marathon) at [email protected]:40447 Jun 7 17:37:16 m01 marathon: [2020-06-07 17:37:16,444] INFO task reconciliation has finished (mesosphere.marathon.MarathonSchedulerActor:marathon-akka.actor.default-dispatcher-2)
Jun 7 17:37:16 m01 marathon: [2020-06-07 17:37:16,459] INFO Message [mesosphere.marathon.MarathonSchedulerActor$TasksReconciled$] from Actor[akka://marathon/user/MarathonScheduler/$a#-463341905] to Actor[akka://marathon/deadLetters] was not delivered. [1] dead letters encountered. If this is not an expected behavior, then [Actor[akka://marathon/deadLetters]] may have terminated unexpectedly, This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. (akka.actor.DeadLetterActorRef:marathon-akka.actor.default-dispatcher-8) Jun 7 17:37:16 m01 marathon: [2020-06-07 17:37:16,502] INFO Prompting Mesos for a heartbeat via explicit task reconciliation (mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor$$anon$1:marathon-akka.actor.default-dispatcher-5) Jun 7 17:37:16 m01 mesos-master[10661]: I0607 17:37:16.506299 10687 master.cpp:8889] Performing explicit task state reconciliation for 1 tasks of framework f2318310-8c7b-438c-9a9d-48fdf1cd0406-0000 (marathon) at [email protected]:40447 Jun 7 17:37:16 m01 marathon: [2020-06-07 17:37:16,513] INFO Received fake heartbeat task-status update (mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor:Thread-14) Jun 7 17:37:31 m01 marathon: [2020-06-07 17:37:31,012] INFO Killing overdue instances: (mesosphere.marathon.core.task.jobs.impl.OverdueInstancesActor$Support:scala-execution-context-global-54) Jun 7 17:37:31 m01 marathon: [2020-06-07 17:37:31,018] INFO Kill and forget following instances for reason Overdue: (mesosphere.marathon.core.task.termination.impl.KillServ
@f1-outsourcing, could you attach the complete logs of Marathon 1.8 from when you start until you made a request to v2/info
.
Jun 7 17:40:43 m01 marathon: No start hook file found ($HOOK_MARATHON_START). Proceeding with the start script.
Jun 7 17:40:46 m01 marathon: [2020-06-07 17:40:46,247] INFO Successfully loaded libmesos: version 1.10.0 (mesosphere.marathon.MarathonApp:main)
Jun 7 17:40:48 m01 marathon: [2020-06-07 17:40:48,481] INFO Slf4jLogger started (akka.event.slf4j.Slf4jLogger:marathon-akka.actor.default-dispatcher-6)
Jun 7 17:40:50 m01 marathon: [2020-06-07 17:40:50,642] INFO Starting heartbeat actor (mesosphere.marathon.core.heartbeat.HeartbeatActor:marathon-akka.actor.default-dispatcher-6)
Jun 7 17:40:51 m01 marathon: [2020-06-07 17:40:51,005] INFO Running in ZooKeeper 3.4.x compatibility mode (org.apache.curator.utils.Compatibility:main)
Jun 7 17:40:51 m01 marathon: [2020-06-07 17:40:51,113] INFO Starting (org.apache.curator.framework.imps.CuratorFrameworkImpl:main)
Jun 7 17:40:51 m01 marathon: [2020-06-07 17:40:51,136] INFO Client environment:zookeeper.version=3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0, built on 11/01/2017 18:06 GMT (org.apache.zookeeper.ZooKeeper:main)
Jun 7 17:40:51 m01 marathon: [2020-06-07 17:40:51,139] INFO Client environment:host.name=m01.local (org.apache.zookeeper.ZooKeeper:main)
Jun 7 17:40:51 m01 marathon: [2020-06-07 17:40:51,141] INFO Client environment:java.version=1.8.0_252 (org.apache.zookeeper.ZooKeeper:main)
Jun 7 17:40:51 m01 marathon: [2020-06-07 17:40:51,143] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper:main)
Jun 7 17:40:51 m01 marathon: [2020-06-07 17:40:51,147] INFO Client environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.x86_64/jre (org.apache.zookeeper.ZooKeeper:main)
Jun 7 17:40:51 m01 marathon: [2020-06-07 17:40:51,150] INFO Client environment:java.class.path=/home/software/marathon-1.8.222-86475ddac/lib/mesosphere.marathon.marathon-1.8.222.jar:/home/software/marathon-1.8.222-86475ddac/lib/mesosphere.marathon.plugin-interface-1.8.222.jar:/home/software/marathon-1.8.222-86475ddac/lib/commons-codec.commons-codec-1.9.jar:/home/software/marathon-1.8.222-86475ddac/lib/org.eclipse.jetty.jetty-http-9.4.8.v20171121.jar:/home/software/marathon-1.8.222-86475ddac/lib/net.logstash.logback.logstash-logback-encoder-4.9.jar:/home/software/marathon-1.8.222-86475ddac/lib/org.apache.yetus.audience-annotations-0.5.0.jar:/home/software/marathon-1.8.222-86475ddac/lib/org.scala-lang.scala-reflect-2.12.7.jar:/home/software/marathon-1.8.222-86475ddac/lib/com.fasterxml.jackson.module.jackson-module-jaxb-annotations-2.9.5.jar:/home/software/marathon-1.8.222-86475ddac/lib/org.scalamacros.resetallattrs_2.12-1.0.0.jar:/home/software/marathon-1.8.222-86475ddac/lib/org.glassfish.jersey.core.jersey-server-2.27.jar:/home/software/marathon-1.8.222-86475ddac/lib/io.netty.netty-3.10.5.Final.jar:/home/software/marathon-1.8.222-86475ddac/lib/com.typesafe.akka.akka-protobuf_2.12-2.5.23.jar:/home/software/marathon-1.8.222-86475ddac/lib/javax.xml.bind.jaxb-api-2.3.1.jar:/home/software/marathon-1.8.222-86475ddac/lib/com.fasterxml.jackson.module.jackson-module-paranamer-2.9.5.jar:/home/software/marathon-1.8.222-86475ddac/lib/com.lightbend.akka.akka-stream-alpakka-s3_2.12-0.14.jar:/home/software/marathon-1.8.222-86475ddac/lib/com.fasterxml.jackson.module.jackson-module-scala_2.12-2.9.5.jar:/home/software/marathon-1.8.222-86475ddac/lib/com.typesafe.play.play-json_2.12-2.6.7.jar:/home/software/marathon-1.8.222-86475ddac/lib/org.glassfish.jersey.media.jersey-media-multipart-2.27.jar:/home/software/marathon-1.8.222-86475ddac/lib/org.glassfish.hk2.hk2-api-2.5.0-b42.jar:/home/software/marathon-1.8.222-86475ddac/lib/org.eclipse.jetty.jetty-servlet-9.4.8.v20171121.jar:/home/software/marathon-1.8.222-86475ddac/lib/org.ap
Jun 7 17:40:51 m01 marathon: [2020-06-07 17:40:51,150] INFO Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper:main)
Jun 7 17:40:51 m01 marathon: [2020-06-07 17:40:51,150] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper:main)
Jun 7 17:40:51 m01 marathon: [2020-06-07 17:40:51,152] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper:main)
Jun 7 17:40:51 m01 marathon: [2020-06-07 17:40:51,152] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper:main)
Jun 7 17:40:51 m01 marathon: [2020-06-07 17:40:51,152] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper:main)
Jun 7 17:40:51 m01 marathon: [2020-06-07 17:40:51,153] INFO Client environment:os.version=3.10.0-1127.10.1.el7.x86_64 (org.apache.zookeeper.ZooKeeper:main)
Jun 7 17:40:51 m01 marathon: [2020-06-07 17:40:51,153] INFO Client environment:user.name=marathon (org.apache.zookeeper.ZooKeeper:main)
Jun 7 17:40:51 m01 marathon: [2020-06-07 17:40:51,156] INFO Client environment:user.home=/home/marathon (org.apache.zookeeper.ZooKeeper:main)
Jun 7 17:40:51 m01 marathon: [2020-06-07 17:40:51,156] INFO Client environment:user.dir=/home/software/marathon-1.8.222-86475ddac (org.apache.zookeeper.ZooKeeper:main)
Jun 7 17:40:51 m01 marathon: [2020-06-07 17:40:51,166] INFO Initiating client connection, connectString=m01.local:2181 sessionTimeout=10000 watcher=org.apache.curator.ConnectionState@516ebdf8 (org.apache.zookeeper.ZooKeeper:main)
Jun 7 17:40:51 m01 marathon: [2020-06-07 17:40:51,259] INFO Default schema (org.apache.curator.framework.imps.CuratorFrameworkImpl:main)
Jun 7 17:40:51 m01 marathon: [2020-06-07 17:40:51,262] INFO Opening socket connection to server m01.local/192.168.10.151:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
Jun 7 17:40:51 m01 marathon: [2020-06-07 17:40:51,286] INFO Socket connection established to m01.local/192.168.10.151:2181, initiating session (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
Jun 7 17:40:51 m01 marathon: [2020-06-07 17:40:51,324] INFO Session establishment complete on server m01.local/192.168.10.151:2181, sessionid = 0x1728f7057480004, negotiated timeout = 10000 (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
Jun 7 17:40:51 m01 marathon: [2020-06-07 17:40:51,344] INFO State change: CONNECTED (org.apache.curator.framework.state.ConnectionStateManager:main-EventThread)
Jun 7 17:40:51 m01 marathon: [2020-06-07 17:40:51,426] WARN The version of ZooKeeper being used doesn't support Container nodes. CreateMode.PERSISTENT will be used instead. (org.apache.curator.utils.ZKPaths:main)
Jun 7 17:40:52 m01 marathon: [2020-06-07 17:40:52,383] INFO Started TaskTrackerUpdateStepsProcessorImpl with steps:
Jun 7 17:40:52 m01 marathon: * continueOnError(notifyHealthCheckManager)
Jun 7 17:40:52 m01 marathon: * continueOnError(notifyRateLimiter)
Jun 7 17:40:52 m01 marathon: * continueOnError(notifyLaunchQueue)
Jun 7 17:40:52 m01 marathon: * continueOnError(postTaskStatusEvent)
Jun 7 17:40:52 m01 marathon: * continueOnError(scaleApp) (mesosphere.marathon.core.task.tracker.impl.InstanceTrackerUpdateStepProcessorImpl:main)
Jun 7 17:40:53 m01 marathon: [2020-06-07 17:40:53,128] INFO Loading plugins implementing 'mesosphere.marathon.plugin.task.RunSpecTaskProcessor' from these urls: [] (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
Jun 7 17:40:53 m01 marathon: [2020-06-07 17:40:53,168] INFO Found 0 plugins. (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
Jun 7 17:40:53 m01 marathon: [2020-06-07 17:40:53,173] INFO Loading plugins implementing 'mesosphere.marathon.plugin.validation.RunSpecValidator' from these urls: [] (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
Jun 7 17:40:53 m01 marathon: [2020-06-07 17:40:53,175] INFO Found 0 plugins. (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
Jun 7 17:40:53 m01 marathon: [2020-06-07 17:40:53,316] INFO Loading plugins implementing 'mesosphere.marathon.plugin.auth.Authenticator' from these urls: [] (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
Jun 7 17:40:53 m01 marathon: [2020-06-07 17:40:53,322] INFO Found 0 plugins. (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
Jun 7 17:40:53 m01 marathon: [2020-06-07 17:40:53,333] INFO Loading plugins implementing 'mesosphere.marathon.plugin.auth.Authorizer' from these urls: [] (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
Jun 7 17:40:53 m01 marathon: [2020-06-07 17:40:53,336] INFO Found 0 plugins. (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
Jun 7 17:40:53 m01 marathon: [2020-06-07 17:40:53,372] INFO Loading plugins implementing 'mesosphere.marathon.plugin.scheduler.SchedulerPlugin' from these urls: [] (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
Jun 7 17:40:53 m01 marathon: [2020-06-07 17:40:53,376] INFO Found 0 plugins. (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
Jun 7 17:40:53 m01 marathon: [2020-06-07 17:40:53,537] INFO Started status update processor (mesosphere.marathon.core.task.update.impl.TaskStatusUpdateProcessorImpl:main)
Jun 7 17:40:53 m01 marathon: [2020-06-07 17:40:53,628] INFO All actors suspended:
Jun 7 17:40:53 m01 marathon: * Actor[akka://marathon/user/deploymentManager#1106232599]
Jun 7 17:40:53 m01 marathon: * Actor[akka://marathon/user/offerMatcherManager#-1876253398]
Jun 7 17:40:53 m01 marathon: * Actor[akka://marathon/user/taskKillServiceActor#-778749779]
Jun 7 17:40:53 m01 marathon: * Actor[akka://marathon/user/rateLimiter#-2044772544]
Jun 7 17:40:53 m01 marathon: * Actor[akka://marathon/user/killOverdueStagedTasks#1783287740]
Jun 7 17:40:53 m01 marathon: * Actor[akka://marathon/user/launchQueue#465459569]
Jun 7 17:40:53 m01 marathon: * Actor[akka://marathon/user/expungeOverdueLostTasks#-1395865443]
Jun 7 17:40:53 m01 marathon: * Actor[akka://marathon/user/reviveOffers#-363088573]
Jun 7 17:40:53 m01 marathon: * Actor[akka://marathon/user/instanceTracker#-1547106758]
Jun 7 17:40:53 m01 marathon: * Actor[akka://marathon/user/offerMatcherLaunchTokens#790335717] (mesosphere.marathon.core.leadership.impl.LeadershipCoordinatorActor:marathon-akka.actor.default-dispatcher-5)
Jun 7 17:40:53 m01 marathon: [2020-06-07 17:40:53,671] INFO Created Mesos heartbeat monitor for scheduler mesosphere.marathon.MarathonScheduler@26221bad (mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor:main)
Jun 7 17:40:53 m01 marathon: [2020-06-07 17:40:53,707] INFO Created Mesos heartbeat monitor for scheduler mesosphere.marathon.MarathonScheduler@26221bad (mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor:main)
Jun 7 17:40:53 m01 marathon: [2020-06-07 17:40:53,733] INFO History actor ready (mesosphere.marathon.core.history.impl.HistoryActor:marathon-akka.actor.default-dispatcher-2)
Jun 7 17:40:53 m01 marathon: [2020-06-07 17:40:53,881] INFO Logging initialized @9789ms to org.eclipse.jetty.util.log.Slf4jLog (org.eclipse.jetty.util.log:main)
Jun 7 17:40:53 m01 marathon: [2020-06-07 17:40:53,960] INFO Adding HTTP support. (mesosphere.marathon.api.HttpModule:main)
Jun 7 17:40:53 m01 marathon: [2020-06-07 17:40:53,967] INFO No HTTPS support configured. (mesosphere.marathon.api.HttpModule:main)
Jun 7 17:40:54 m01 marathon: [2020-06-07 17:40:54,118] INFO Starting Marathon 1.8.222/86475ddac52b6aa05697be7ea590dde9d288496b with --hostname m01.local --zk zk://m01.local:2181/marathon --http_port 8081 --master zk://m01.local:2181/mesos --logging_level info --mesos_user root (mesosphere.marathon.MarathonApp:main)
Jun 7 17:40:54 m01 marathon: [2020-06-07 17:40:54,280] INFO Loading plugins implementing 'mesosphere.marathon.plugin.http.HttpRequestHandler' from these urls: [] (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
Jun 7 17:40:54 m01 marathon: [2020-06-07 17:40:54,287] INFO Found 0 plugins. (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
Jun 7 17:40:55 m01 marathon: [2020-06-07 17:40:55,617] INFO Now standing by. Closing existing handles and rejecting new. (mesosphere.marathon.core.event.impl.stream.HttpEventStreamActor:marathon-akka.actor.default-dispatcher-6)
Jun 7 17:40:55 m01 marathon: [2020-06-07 17:40:55,711] INFO Starting up HttpServer. (mesosphere.marathon.api.MarathonHttpService:MarathonHttpService STARTING)
Jun 7 17:40:55 m01 marathon: [2020-06-07 17:40:55,717] INFO jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8 (org.eclipse.jetty.server.Server:MarathonHttpService STARTING)
Jun 7 17:40:55 m01 marathon: [2020-06-07 17:40:55,720] INFO Starting up (mesosphere.marathon.MarathonSchedulerService:MarathonSchedulerService)
Jun 7 17:40:55 m01 marathon: [2020-06-07 17:40:55,724] INFO Beginning run (mesosphere.marathon.MarathonSchedulerService:MarathonSchedulerService)
Jun 7 17:40:55 m01 marathon: [2020-06-07 17:40:55,782] INFO Starting leader latch (mesosphere.marathon.core.election.CuratorElectionStream$:MarathonSchedulerService)
Jun 7 17:40:55 m01 marathon: [2020-06-07 17:40:55,862] INFO Leader watch loop 1 retrying; waiting for latch initialization. (mesosphere.marathon.core.election.CuratorElectionStream$:pool-1-thread-1)
Jun 7 17:40:55 m01 marathon: [2020-06-07 17:40:55,889] INFO Leader watch loop 1 retrying; waiting for latch initialization. (mesosphere.marathon.core.election.CuratorElectionStream$:pool-1-thread-1)
Jun 7 17:40:55 m01 marathon: [2020-06-07 17:40:55,903] INFO Leader watch loop 1 retrying; waiting for latch initialization. (mesosphere.marathon.core.election.CuratorElectionStream$:pool-1-thread-1)
Jun 7 17:40:55 m01 marathon: [2020-06-07 17:40:55,934] INFO Leader watch loop 1 retrying; waiting for latch initialization. (mesosphere.marathon.core.election.CuratorElectionStream$:pool-1-thread-1)
Jun 7 17:40:56 m01 marathon: [2020-06-07 17:40:56,037] INFO Leader won: m01.local:8081 (mesosphere.marathon.core.election.CuratorElectionStream$:marathon-akka.actor.default-dispatcher-6)
Jun 7 17:40:56 m01 marathon: [2020-06-07 17:40:56,042] INFO As new leader running the driver (mesosphere.marathon.MarathonSchedulerService:pool-1-thread-1)
Jun 7 17:40:57 m01 marathon: [2020-06-07 17:40:57,496] INFO No migration necessary, no version stored (mesosphere.marathon.storage.migration.Migration:scala-execution-context-global-40)
Jun 7 17:40:57 m01 marathon: [2020-06-07 17:40:57,626] INFO Migration successfully applied for version Version(18, 200, 0, PERSISTENCE_STORE) (mesosphere.marathon.storage.migration.Migration:scala-execution-context-global-40)
Jun 7 17:40:57 m01 marathon: [2020-06-07 17:40:57,736] INFO Call preDriverStarts callbacks on (mesosphere.marathon.MarathonSchedulerService:pool-1-thread-1)
Jun 7 17:40:57 m01 marathon: [2020-06-07 17:40:57,737] INFO Finished preDriverStarts callbacks (mesosphere.marathon.MarathonSchedulerService:pool-1-thread-1)
Jun 7 17:40:57 m01 marathon: [2020-06-07 17:40:57,758] INFO ExpungeOverdueLostTasksActor has started (mesosphere.marathon.core.task.jobs.impl.ExpungeOverdueLostTasksActor:marathon-akka.actor.default-dispatcher-5)
Jun 7 17:40:57 m01 marathon: [2020-06-07 17:40:57,760] INFO started RateLimiterActor (mesosphere.marathon.core.launchqueue.impl.RateLimiterActor:marathon-akka.actor.default-dispatcher-4)
Jun 7 17:40:57 m01 marathon: [2020-06-07 17:40:57,832] INFO All actors active:
Jun 7 17:40:57 m01 marathon: * Actor[akka://marathon/user/deploymentManager#1106232599]
Jun 7 17:40:57 m01 marathon: * Actor[akka://marathon/user/offerMatcherManager#-1876253398]
Jun 7 17:40:57 m01 marathon: * Actor[akka://marathon/user/taskKillServiceActor#-778749779]
Jun 7 17:40:57 m01 marathon: * Actor[akka://marathon/user/rateLimiter#-2044772544]
Jun 7 17:40:57 m01 marathon: * Actor[akka://marathon/user/killOverdueStagedTasks#1783287740]
Jun 7 17:40:57 m01 marathon: * Actor[akka://marathon/user/launchQueue#465459569]
Jun 7 17:40:57 m01 marathon: * Actor[akka://marathon/user/expungeOverdueLostTasks#-1395865443]
Jun 7 17:40:57 m01 marathon: * Actor[akka://marathon/user/reviveOffers#-363088573]
Jun 7 17:40:57 m01 marathon: * Actor[akka://marathon/user/instanceTracker#-1547106758]
Jun 7 17:40:57 m01 marathon: * Actor[akka://marathon/user/offerMatcherLaunchTokens#790335717] (mesosphere.marathon.core.leadership.impl.LeadershipCoordinatorActor:marathon-akka.actor.default-dispatcher-7)
Jun 7 17:40:57 m01 marathon: [2020-06-07 17:40:57,845] INFO InstanceTrackerActor is starting. Instances loading initiated. (mesosphere.marathon.core.task.tracker.impl.InstanceTrackerActor:marathon-akka.actor.default-dispatcher-4)
Jun 7 17:40:57 m01 marathon: [2020-06-07 17:40:57,986] INFO Loaded 0 instances (mesosphere.marathon.core.task.tracker.impl.InstancesLoaderImpl:scala-execution-context-global-40)
Jun 7 17:40:58 m01 marathon: [2020-06-07 17:40:58,006] WARN No frameworkId could be read and no instances are defined. This will result in a new frameworkId (mesosphere.marathon.MesosSchedulerDriverFactory$:pool-1-thread-1)
Jun 7 17:40:58 m01 marathon: [2020-06-07 17:40:58,010] INFO Create new Scheduler Driver with frameworkId: None and scheduler mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor@1850d3c8 (mesosphere.marathon.MarathonSchedulerDriver$:pool-1-thread-1)
Jun 7 17:40:58 m01 marathon: [2020-06-07 17:40:58,021] INFO Instances loading complete. (mesosphere.marathon.core.task.tracker.impl.InstanceTrackerActor:marathon-akka.actor.default-dispatcher-6)
Jun 7 17:40:58 m01 marathon: [2020-06-07 17:40:58,024] INFO Loaded 0 instances. (mesosphere.marathon.core.task.tracker.impl.InstanceTrackerActor:marathon-akka.actor.default-dispatcher-6)
Jun 7 17:40:58 m01 marathon: [2020-06-07 17:40:58,126] INFO PARTITION_AWARE feature enabled. (mesosphere.marathon.MarathonSchedulerDriver$:pool-1-thread-1)
Jun 7 17:40:58 m01 marathon: [2020-06-07 17:40:58,137] INFO REGION_AWARE feature enabled (mesosphere.marathon.MarathonSchedulerDriver$:pool-1-thread-1)
Jun 7 17:40:58 m01 marathon: [2020-06-07 17:40:58,281] WARN A provider mesosphere.marathon.api.v2.QueueResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider mesosphere.marathon.api.v2.QueueResource will be ignored. (org.glassfish.jersey.internal.inject.Providers:MarathonHttpService STARTING)
Jun 7 17:40:58 m01 marathon: [2020-06-07 17:40:58,284] WARN A provider mesosphere.marathon.api.v2.GroupsResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider mesosphere.marathon.api.v2.GroupsResource will be ignored. (org.glassfish.jersey.internal.inject.Providers:MarathonHttpService STARTING)
Jun 7 17:40:58 m01 marathon: [2020-06-07 17:40:58,286] WARN A provider mesosphere.marathon.api.v2.InfoResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider mesosphere.marathon.api.v2.InfoResource will be ignored. (org.glassfish.jersey.internal.inject.Providers:MarathonHttpService STARTING)
Jun 7 17:40:58 m01 marathon: [2020-06-07 17:40:58,289] WARN A provider mesosphere.marathon.api.v2.TasksResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider mesosphere.marathon.api.v2.TasksResource will be ignored. (org.glassfish.jersey.internal.inject.Providers:MarathonHttpService STARTING)
Jun 7 17:40:58 m01 marathon: [2020-06-07 17:40:58,290] WARN A provider mesosphere.marathon.api.v2.LeaderResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider mesosphere.marathon.api.v2.LeaderResource will be ignored. (org.glassfish.jersey.internal.inject.Providers:MarathonHttpService STARTING)
Jun 7 17:40:58 m01 marathon: [2020-06-07 17:40:58,291] WARN A provider mesosphere.marathon.api.v2.PodsResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider mesosphere.marathon.api.v2.PodsResource will be ignored. (org.glassfish.jersey.internal.inject.Providers:MarathonHttpService STARTING)
Jun 7 17:40:58 m01 marathon: [2020-06-07 17:40:58,296] WARN A provider mesosphere.marathon.api.v2.PluginsResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider mesosphere.marathon.api.v2.PluginsResource will be ignored. (org.glassfish.jersey.internal.inject.Providers:MarathonHttpService STARTING)
Jun 7 17:40:58 m01 marathon: [2020-06-07 17:40:58,297] WARN A provider mesosphere.marathon.api.SystemResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider mesosphere.marathon.api.SystemResource will be ignored. (org.glassfish.jersey.internal.inject.Providers:MarathonHttpService STARTING)
Jun 7 17:40:58 m01 marathon: [2020-06-07 17:40:58,297] WARN A provider mesosphere.marathon.api.v2.DeploymentsResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider mesosphere.marathon.api.v2.DeploymentsResource will be ignored. (org.glassfish.jersey.internal.inject.Providers:MarathonHttpService STARTING)
Jun 7 17:40:58 m01 marathon: [2020-06-07 17:40:58,303] WARN A provider mesosphere.marathon.api.v2.AppsResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider mesosphere.marathon.api.v2.AppsResource will be ignored. (org.glassfish.jersey.internal.inject.Providers:MarathonHttpService STARTING)
Jun 7 17:40:58 m01 marathon: [2020-06-07 17:40:58,836] INFO Suppress: There are no pending instances right now. (mesosphere.marathon.core.launchqueue.impl.ReviveOffersStreamLogic$:marathon-akka.actor.default-dispatcher-4)
Jun 7 17:40:58 m01 marathon: [2020-06-07 17:40:58,841] INFO Sending suppress (mesosphere.marathon.core.launchqueue.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-4)
Jun 7 17:40:58 m01 marathon: [2020-06-07 17:40:58,892] INFO Heartbeat actor activated (mesosphere.marathon.core.heartbeat.HeartbeatActor:marathon-akka.actor.default-dispatcher-4)
Jun 7 17:40:58 m01 marathon: 2020-06-07 17:40:58,905:11304(0x7f257edc9700):ZOO_INFO@log_env@753: Client environment:zookeeper.version=zookeeper C client 3.4.8
Jun 7 17:40:58 m01 marathon: 2020-06-07 17:40:58,907:11304(0x7f257edc9700):ZOO_INFO@log_env@757: Client environment:host.name=m01.local
Jun 7 17:40:58 m01 marathon: 2020-06-07 17:40:58,909:11304(0x7f257edc9700):ZOO_INFO@log_env@764: Client environment:os.name=Linux
Jun 7 17:40:58 m01 marathon: 2020-06-07 17:40:58,909:11304(0x7f257edc9700):ZOO_INFO@log_env@765: Client environment:os.arch=3.10.0-1127.10.1.el7.x86_64
Jun 7 17:40:58 m01 marathon: 2020-06-07 17:40:58,909:11304(0x7f257edc9700):ZOO_INFO@log_env@766: Client environment:os.version=#1 SMP Wed Jun 3 14:28:03 UTC 2020
Jun 7 17:40:58 m01 marathon: 2020-06-07 17:40:58,909:11304(0x7f257edc9700):ZOO_INFO@log_env@774: Client environment:user.name=(null)
Jun 7 17:40:58 m01 marathon: 2020-06-07 17:40:58,909:11304(0x7f257edc9700):ZOO_INFO@log_env@782: Client environment:user.home=/home/marathon
Jun 7 17:40:58 m01 marathon: 2020-06-07 17:40:58,909:11304(0x7f257edc9700):ZOO_INFO@log_env@794: Client environment:user.dir=/home/software/marathon-1.8.222-86475ddac
Jun 7 17:40:58 m01 marathon: 2020-06-07 17:40:58,909:11304(0x7f257edc9700):ZOO_INFO@zookeeper_init@827: Initiating client connection, host=m01.local:2181 sessionTimeout=10000 watcher=0x7f2585e994e0 sessionId=0 sessionPasswd=<null> context=0x7f25c5b14930 flags=0
Jun 7 17:40:58 m01 marathon: I0607 17:40:58.921898 11398 sched.cpp:239] Version: 1.10.0
Jun 7 17:40:58 m01 marathon: [2020-06-07 17:40:58,936] INFO Starting scheduler actor (mesosphere.marathon.MarathonSchedulerActor:marathon-akka.actor.default-dispatcher-3)
Jun 7 17:40:58 m01 marathon: [2020-06-07 17:40:58,936] INFO Became active. Accepting event streaming requests. (mesosphere.marathon.core.event.impl.stream.HttpEventStreamActor:marathon-akka.actor.default-dispatcher-2)
Jun 7 17:40:58 m01 zookeeper: 2020-06-07 17:40:58,956 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@822] - Connection request from old client /192.168.10.151:37566; will be dropped if server is in r-o mode
Jun 7 17:40:58 m01 marathon: 2020-06-07 17:40:58,959:11304(0x7f257bcc2700):ZOO_INFO@check_events@1764: initiated connection to server [192.168.10.151:2181]
Jun 7 17:40:58 m01 marathon: [2020-06-07 17:40:58,965] INFO Scheduler actor ready (mesosphere.marathon.MarathonSchedulerActor:marathon-akka.actor.default-dispatcher-3)
Jun 7 17:40:58 m01 marathon: 2020-06-07 17:40:58,990:11304(0x7f257bcc2700):ZOO_INFO@check_events@1811: session establishment complete on server [192.168.10.151:2181], sessionId=0x1728f7057480005, negotiated timeout=10000
Jun 7 17:40:58 m01 marathon: I0607 17:40:58.993643 11406 group.cpp:341] Group process (zookeeper-group(1)@192.168.10.151:36941) connected to ZooKeeper
Jun 7 17:40:58 m01 marathon: I0607 17:40:58.995862 11406 group.cpp:831] Syncing group operations: queue size (joins, cancels, datas) = (0, 0, 0)
Jun 7 17:40:58 m01 marathon: I0607 17:40:58.998025 11406 group.cpp:419] Trying to create path '/mesos' in ZooKeeper
Jun 7 17:40:59 m01 marathon: I0607 17:40:59.003587 11410 detector.cpp:152] Detected a new leader: (id='0')
Jun 7 17:40:59 m01 marathon: I0607 17:40:59.006004 11406 group.cpp:700] Trying to get '/mesos/json.info_0000000000' in ZooKeeper
Jun 7 17:40:59 m01 marathon: I0607 17:40:59.035372 11409 zookeeper.cpp:262] A new leading master ([email protected]:5050) is detected
Jun 7 17:40:59 m01 marathon: I0607 17:40:59.038015 11404 sched.cpp:343] New master detected at [email protected]:5050
Jun 7 17:40:59 m01 marathon: I0607 17:40:59.041571 11404 sched.cpp:363] No credentials provided. Attempting to register without authentication
Jun 7 17:40:59 m01 mesos-master[11203]: I0607 17:40:59.046126 11233 master.cpp:2901] Received SUBSCRIBE call for framework 'marathon' at [email protected]:36941
Jun 7 17:40:59 m01 mesos-master[11203]: I0607 17:40:59.046856 11233 master.cpp:2983] Subscribing framework marathon with checkpointing enabled and capabilities [ PARTITION_AWARE, REGION_AWARE ]
Jun 7 17:40:59 m01 mesos-master[11203]: I0607 17:40:59.051831 11233 master.cpp:9897] Adding framework f5d67e06-6600-4fb9-94dc-a878be2563be-0000 (marathon) at [email protected]:36941 with roles { } suppressed
Jun 7 17:40:59 m01 mesos-master[11203]: I0607 17:40:59.056349 11228 hierarchical.cpp:767] Added framework f5d67e06-6600-4fb9-94dc-a878be2563be-0000
Jun 7 17:40:59 m01 marathon: I0607 17:40:59.060653 11403 sched.cpp:751] Framework registered with f5d67e06-6600-4fb9-94dc-a878be2563be-0000
Jun 7 17:40:59 m01 marathon: [2020-06-07 17:40:59,095] INFO Registered heartbeat monitor (mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor:Thread-12)
Jun 7 17:40:59 m01 marathon: [2020-06-07 17:40:59,098] INFO Registered as f5d67e06-6600-4fb9-94dc-a878be2563be-0000 to master 'f5d67e06-6600-4fb9-94dc-a878be2563be' (mesosphere.marathon.MarathonScheduler:Thread-12)
Jun 7 17:40:59 m01 marathon: [2020-06-07 17:40:59,100] INFO Mesos Master version 1.10.0 (mesosphere.marathon.MarathonScheduler:Thread-12)
Jun 7 17:40:59 m01 marathon: [2020-06-07 17:40:59,638] WARN The following warnings have been detected: WARNING: The (sub)resource method redirectUI in mesosphere.marathon.api.SystemResource contains empty path annotation.
Jun 7 17:40:59 m01 marathon: (org.glassfish.jersey.internal.Errors:MarathonHttpService STARTING)
Jun 7 17:40:59 m01 marathon: [2020-06-07 17:40:59,655] INFO Started o.e.j.s.ServletContextHandler@65b2a783{/,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler:MarathonHttpService STARTING)
Jun 7 17:40:59 m01 marathon: [2020-06-07 17:40:59,690] INFO Started ServerConnector@4f89a524{HTTP/1.1,[http/1.1]}{0.0.0.0:8081} (org.eclipse.jetty.server.AbstractConnector:MarathonHttpService STARTING)
Jun 7 17:40:59 m01 marathon: [2020-06-07 17:40:59,693] INFO Started @15602ms (org.eclipse.jetty.server.Server:MarathonHttpService STARTING)
Jun 7 17:40:59 m01 marathon: [2020-06-07 17:40:59,696] INFO All services up and running. (mesosphere.marathon.MarathonApp:main)
Jun 7 17:41:13 m01 marathon: [2020-06-07 17:41:13,833] INFO initiate task reconciliation (mesosphere.marathon.MarathonSchedulerActor:marathon-akka.actor.default-dispatcher-9)
Jun 7 17:41:13 m01 marathon: [2020-06-07 17:41:13,854] INFO Requesting task reconciliation with the Mesos master (mesosphere.marathon.SchedulerActions:scheduler-actions-thread-0)
Jun 7 17:41:13 m01 mesos-master[11203]: I0607 17:41:13.858621 11227 master.cpp:8846] Performing implicit task state reconciliation for framework f5d67e06-6600-4fb9-94dc-a878be2563be-0000 (marathon) at [email protected]:36941
Jun 7 17:41:13 m01 marathon: [2020-06-07 17:41:13,864] INFO task reconciliation has finished (mesosphere.marathon.MarathonSchedulerActor:marathon-akka.actor.default-dispatcher-4)
Jun 7 17:41:13 m01 marathon: [2020-06-07 17:41:13,879] INFO Message [mesosphere.marathon.MarathonSchedulerActor$TasksReconciled$] from Actor[akka://marathon/user/MarathonScheduler/$a#1746491390] to Actor[akka://marathon/deadLetters] was not delivered. [1] dead letters encountered. If this is not an expected behavior, then [Actor[akka://marathon/deadLetters]] may have terminated unexpectedly, This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. (akka.actor.DeadLetterActorRef:marathon-akka.actor.default-dispatcher-7)
Jun 7 17:41:13 m01 marathon: [2020-06-07 17:41:13,910] INFO Prompting Mesos for a heartbeat via explicit task reconciliation (mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor$$anon$1:marathon-akka.actor.default-dispatcher-7)
Jun 7 17:41:13 m01 mesos-master[11203]: I0607 17:41:13.914615 11228 master.cpp:8889] Performing explicit task state reconciliation for 1 tasks of framework f5d67e06-6600-4fb9-94dc-a878be2563be-0000 (marathon) at [email protected]:36941
Jun 7 17:41:13 m01 marathon: [2020-06-07 17:41:13,924] INFO Received fake heartbeat task-status update (mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor:Thread-13)
Jun 7 17:41:28 m01 marathon: [2020-06-07 17:41:28,939] INFO Prompting Mesos for a heartbeat via explicit task reconciliation (mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor$$anon$1:marathon-akka.actor.default-dispatcher-4)
Jun 7 17:41:28 m01 mesos-master[11203]: I0607 17:41:28.946494 11229 master.cpp:8889] Performing explicit task state reconciliation for 1 tasks of framework f5d67e06-6600-4fb9-94dc-a878be2563be-0000 (marathon) at [email protected]:36941
Jun 7 17:41:28 m01 marathon: [2020-06-07 17:41:28,950] INFO Received fake heartbeat task-status update (mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor:Thread-14)
@f1-outsourcing, could you attach the complete logs of Marathon 1.8 from when you start until you made a request to
v2/info
.
Is this useful?
Is this being looked at still?
If your strategy at D2iQ/mesosphere is to give 'shitty' support to marathon, because you want to push people into using DCOS. You should consider there is a flip side to that approach, I perceive this as:
-
If you do not have time to resolve this probably simple issue, how can users of DCOS trust you to do a good job maintaining this 1GB blob install you offer? I prefer to run your mesos and marathon on CentOS/RedHat because I trust RedHat to do a better job at back porting bugs than you do. EL is their core business, it is not your core business maintaining packages/distributions. There is no way for me to check if your installed binaries are up to date, and you even take the time to update them.
-
I assume this issue I have, is maybe related to an incompatibility somewhere between mesos and marathon. So that could indicate that in your DCOS blob you are maybe mixing java versions, or using leading edge software releases instead of LTS versions. At least have a policy that is deviates from that of LTS distributions, which again of course does not make sense from a security and stability perspective. The fact that you advertise in mesos that you are cni 0.3 compatible, which you are not[2]. Is only contributing to the perception you do not mind deviating from best practices.
Someone else reported the same issue[1] in March on your JIRA and his work-a-round of downgrading to Marathon 1.7. He also did not get any attention for 6 month's. Whether or not your software is open source, you should attend to such issues quicker, where people need to downgrade so many versions. Otherwise just make this closed source, or withdraw it from public support, so people are not wasting time trying/using it.
[1] https://jira.d2iq.com/browse/MARATHON-8729
[2] https://issues.apache.org/jira/browse/MESOS-7178
Also not working Downgrading mesos to mesos-1.7.2-2.0.1.x86_64 and downgrading marathon to sbt -mem 1024 'run --master m01.local:5050 --zk zk://m01.local:2181/v18207 --framework_name v18207 --http_port 7070'
What about this message:
[info] [2020-07-18 13:14:42,675] INFO Message [mesosphere.marathon.MarathonSchedulerActor$TasksReconciled$] from Actor[akka://marathon/user/MarathonScheduler/$a#-1125671270] to Actor[akka://marathon/deadLetters] was not delivered. [1] dead letters encountered. If this is not an expected behavior, then [Actor[akka://marathon/deadLetters]] may have terminated unexpectedly, This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. (akka.actor.DeadLetterActorRef:marathon-akka.actor.default-dispatcher-8)
debug log of http://test2.local:7070/v2/info
[info] [2020-07-18 13:26:09,845] DEBUG Current State: LaunchTokens:100 OffersWanted:false Matchers:0 OfferQueues:0 UnprocessedOffers:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-9)
[info] [2020-07-18 13:26:09,845] DEBUG Reject Elapsed offers. Unprocessed: 0 Overdue:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-9)
[info] [2020-07-18 13:26:11,788] DEBUG Queued change org.eclipse.jetty.io.ManagedSelector$Accept@2b70db91 on org.eclipse.jetty.io.ManagedSelector@4752a8a1 id=0 keys=0 selected=0 actions=0 (org.eclipse.jetty.io.ManagedSelector:qtp1021656938-31-acceptor-0@654d6319-ServerConnector@2a2843ec{HTTP/1.1,[http/1.1]}{0.0.0.0:7070})
[info] [2020-07-18 13:26:11,789] DEBUG Selector sun.nio.ch.EPollSelectorImpl@3c2dd5d9 woken up from select, 0/0 selected (org.eclipse.jetty.io.ManagedSelector:qtp1021656938-30)
[info] [2020-07-18 13:26:11,790] DEBUG Selector sun.nio.ch.EPollSelectorImpl@3c2dd5d9 processing 0 keys, 1 actions (org.eclipse.jetty.io.ManagedSelector:qtp1021656938-30)
[info] [2020-07-18 13:26:11,790] DEBUG action=org.eclipse.jetty.io.ManagedSelector$Accept@2b70db91 wakeup=false (org.eclipse.jetty.io.ManagedSelector:qtp1021656938-30)
[info] [2020-07-18 13:26:11,790] DEBUG EatWhatYouKill@2d2e8982/SelectorProducer@2b454e29/PRODUCING/ReservedThreadExecutor@4fbe007b{s=0/1,p=0}@SelectorManager@ServerConnector@2a2843ec{HTTP/1.1,[http/1.1]}{0.0.0.0:7070} t=org.eclipse.jetty.io.ManagedSelector$Accept@2b70db91/NON_BLOCKING (org.eclipse.jetty.util.thread.strategy.EatWhatYouKill:qtp1021656938-30)
[info] [2020-07-18 13:26:11,791] DEBUG EatWhatYouKill@2d2e8982/SelectorProducer@2b454e29/PRODUCING/ReservedThreadExecutor@4fbe007b{s=0/1,p=0}@SelectorManager@ServerConnector@2a2843ec{HTTP/1.1,[http/1.1]}{0.0.0.0:7070} PC t=org.eclipse.jetty.io.ManagedSelector$Accept@2b70db91 (org.eclipse.jetty.util.thread.strategy.EatWhatYouKill:qtp1021656938-30)
[info] [2020-07-18 13:26:11,792] DEBUG Queued change org.eclipse.jetty.io.ManagedSelector$CreateEndPoint@1acde28a on org.eclipse.jetty.io.ManagedSelector@4752a8a1 id=0 keys=1 selected=0 actions=0 (org.eclipse.jetty.io.ManagedSelector:qtp1021656938-30)
[info] [2020-07-18 13:26:11,792] DEBUG Forcing selection, actions=1 (org.eclipse.jetty.io.ManagedSelector:qtp1021656938-30)
[info] [2020-07-18 13:26:11,794] DEBUG action=null wakeup=true (org.eclipse.jetty.io.ManagedSelector:qtp1021656938-30)
[info] [2020-07-18 13:26:11,797] DEBUG Selector sun.nio.ch.EPollSelectorImpl@3c2dd5d9 waiting on select (org.eclipse.jetty.io.ManagedSelector:qtp1021656938-30)
[info] [2020-07-18 13:26:11,798] DEBUG Selector sun.nio.ch.EPollSelectorImpl@3c2dd5d9 woken up from select, 0/1 selected (org.eclipse.jetty.io.ManagedSelector:qtp1021656938-30)
[info] [2020-07-18 13:26:11,798] DEBUG Selector sun.nio.ch.EPollSelectorImpl@3c2dd5d9 processing 0 keys, 1 actions (org.eclipse.jetty.io.ManagedSelector:qtp1021656938-30)
[info] [2020-07-18 13:26:11,801] DEBUG action=org.eclipse.jetty.io.ManagedSelector$CreateEndPoint@1acde28a wakeup=false (org.eclipse.jetty.io.ManagedSelector:qtp1021656938-30)
[info] [2020-07-18 13:26:11,801] DEBUG EatWhatYouKill@2d2e8982/SelectorProducer@2b454e29/PRODUCING/ReservedThreadExecutor@4fbe007b{s=0/1,p=0}@SelectorManager@ServerConnector@2a2843ec{HTTP/1.1,[http/1.1]}{0.0.0.0:7070} t=org.eclipse.jetty.io.ManagedSelector$CreateEndPoint@1acde28a/BLOCKING (org.eclipse.jetty.util.thread.strategy.EatWhatYouKill:qtp1021656938-30)
[info] [2020-07-18 13:26:11,803] DEBUG Got ping response for sessionid: 0x1735d68ff190022 after 1ms (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
[info] [2020-07-18 13:26:11,806] DEBUG ReservedThreadExecutor@4fbe007b{s=0/1,p=0}@SelectorManager@ServerConnector@2a2843ec{HTTP/1.1,[http/1.1]}{0.0.0.0:7070} tryExecute EatWhatYouKill@2d2e8982/SelectorProducer@2b454e29/PRODUCING/ReservedThreadExecutor@4fbe007b{s=0/1,p=0}@SelectorManager@ServerConnector@2a2843ec{HTTP/1.1,[http/1.1]}{0.0.0.0:7070} (org.eclipse.jetty.util.thread.ReservedThreadExecutor:qtp1021656938-30)
[info] [2020-07-18 13:26:11,808] DEBUG ReservedThreadExecutor@4fbe007b{s=0/1,p=1}@SelectorManager@ServerConnector@2a2843ec{HTTP/1.1,[http/1.1]}{0.0.0.0:7070} startReservedThread p=1 (org.eclipse.jetty.util.thread.ReservedThreadExecutor:qtp1021656938-30)
[info] [2020-07-18 13:26:11,811] DEBUG queue ReservedThreadExecutor@4fbe007b{s=0/1,p=1}@SelectorManager@ServerConnector@2a2843ec{HTTP/1.1,[http/1.1]}{0.0.0.0:7070}@74e52b3f (org.eclipse.jetty.util.thread.QueuedThreadPool:qtp1021656938-30)
[info] [2020-07-18 13:26:11,820] DEBUG run ReservedThreadExecutor@4fbe007b{s=0/1,p=1}@SelectorManager@ServerConnector@2a2843ec{HTTP/1.1,[http/1.1]}{0.0.0.0:7070}@74e52b3f (org.eclipse.jetty.util.thread.QueuedThreadPool:qtp1021656938-32)
[info] [2020-07-18 13:26:11,820] DEBUG ReservedThreadExecutor@4fbe007b{s=1/1,p=1}@SelectorManager@ServerConnector@2a2843ec{HTTP/1.1,[http/1.1]}{0.0.0.0:7070}@74e52b3f started (org.eclipse.jetty.util.thread.ReservedThreadExecutor:qtp1021656938-32)
[info] [2020-07-18 13:26:11,820] DEBUG ReservedThreadExecutor@4fbe007b{s=1/1,p=0}@SelectorManager@ServerConnector@2a2843ec{HTTP/1.1,[http/1.1]}{0.0.0.0:7070}@74e52b3f waiting (org.eclipse.jetty.util.thread.ReservedThreadExecutor:qtp1021656938-32)
[info] [2020-07-18 13:26:11,821] DEBUG EatWhatYouKill@2d2e8982/SelectorProducer@2b454e29/PRODUCING/ReservedThreadExecutor@4fbe007b{s=1/1,p=0}@SelectorManager@ServerConnector@2a2843ec{HTTP/1.1,[http/1.1]}{0.0.0.0:7070} PEC t=org.eclipse.jetty.io.ManagedSelector$CreateEndPoint@1acde28a (org.eclipse.jetty.util.thread.strategy.EatWhatYouKill:qtp1021656938-30)
[info] [2020-07-18 13:26:11,822] DEBUG queue org.eclipse.jetty.io.ManagedSelector$CreateEndPoint@1acde28a (org.eclipse.jetty.util.thread.QueuedThreadPool:qtp1021656938-30)
[info] [2020-07-18 13:26:11,825] DEBUG run org.eclipse.jetty.io.ManagedSelector$CreateEndPoint@1acde28a (org.eclipse.jetty.util.thread.QueuedThreadPool:qtp1021656938-33)
[info] [2020-07-18 13:26:11,829] DEBUG Forcing selection, actions=0 (org.eclipse.jetty.io.ManagedSelector:qtp1021656938-30)
[info] [2020-07-18 13:26:11,829] DEBUG action=null wakeup=false (org.eclipse.jetty.io.ManagedSelector:qtp1021656938-30)
[info] [2020-07-18 13:26:11,831] DEBUG Selector sun.nio.ch.EPollSelectorImpl@3c2dd5d9 waiting on select (org.eclipse.jetty.io.ManagedSelector:qtp1021656938-30)
[info] [2020-07-18 13:26:11,856] DEBUG SocketChannelEndPoint@423be9fa{/192.168.10.219:49734<->/192.168.10.22:7070,OPEN,fill=-,flush=-,to=24/30000}{io=0/0,kio=0,kro=0}->
tracelog of 1.8 after deploying a task with 1.7
[info] Loading project definition from /home/software/marathon2/project/project
[info] Loading settings for project marathon2-build from plugins.sbt ... [info] Loading project definition from /home/software/marathon2/type-generator/project
[info] Loading settings for project root from build.sbt ... [info] Loading project definition from /home/software/marathon2/project
[info] Loading settings for project marathon from build.sbt ... [info] Set current project to marathon (in build file:/home/software/marathon2/)
[info] Running (fork) mesosphere.marathon.Main --master m01.local:5050 --zk zk://m01.local:2181/marathon --framework_name marathon --http_port 8080 --logging_level trace --hostname test2.local 2020-07-18 15:23:26,028] INFO Successfully loaded libmesos: version 1.10.0 (mesosphere.marathon.MarathonApp:main)
2020-07-18 15:23:26,957] WARN Specifying a Mesos Master connection without a protocol is deprecated and will likely be prohibited in the future. Please specify a protocol (http://, zk://, https://) (mesosphere.marathon.MarathonConf$:main)
2020-07-18 15:23:28,195] INFO Slf4jLogger started (akka.event.slf4j.Slf4jLogger:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:30,345] INFO Starting heartbeat actor (mesosphere.marathon.core.heartbeat.HeartbeatActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:30,784] INFO Running in ZooKeeper 3.4.x compatibility mode (org.apache.curator.utils.Compatibility:main)
2020-07-18 15:23:30,883] INFO Starting (org.apache.curator.framework.imps.CuratorFrameworkImpl:main)
2020-07-18 15:23:30,893] DEBUG Starting (org.apache.curator.CuratorZookeeperClient:main)
2020-07-18 15:23:30,894] DEBUG Starting (org.apache.curator.ConnectionState:main)
2020-07-18 15:23:30,895] DEBUG reset (org.apache.curator.ConnectionState:main)
2020-07-18 15:23:30,914] INFO Client environment:zookeeper.version=3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0, built on 11/01/2017 18:06 GMT (org.apache.zookeeper.ZooKeeper:main)
2020-07-18 15:23:30,915] INFO Client environment:host.name=test2.local (org.apache.zookeeper.ZooKeeper:main)
2020-07-18 15:23:30,917] INFO Client environment:java.version=1.8.0_252 (org.apache.zookeeper.ZooKeeper:main)
2020-07-18 15:23:30,918] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper:main)
2020-07-18 15:23:30,919] INFO Client environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.x86_64/jre (org.apache.zookeeper.ZooKeeper:main)
2020-07-18 15:23:30,920] INFO Client environment:java.class.path=/tmp/sbt_dc4cf587/job-1/target/9370c632/marathon_2.12-1.8.207.jar:/tmp/sbt_dc4cf587/target/fba81147/plugin-interface_2.12-1.8.207.jar:/tmp/sbt_dc4cf587/target/9ce04e34/commons-codec-1.9.jar:/tmp/sbt_dc4cf587/target/9879d6c4/jetty-http-9.4.8.v20171121.jar:/tmp/sbt_dc4cf587/target/4c192f4a/logstash-logback-encoder-4.9.jar:/tmp/sbt_dc4cf587/target/55762d31/audience-annotations-0.5.0.jar:/tmp/sbt_dc4cf587/target/c5a8eb12/scala-reflect.jar:/tmp/sbt_dc4cf587/target/187555de/jackson-module-jaxb-annotations-2.9.5.jar:/tmp/sbt_dc4cf587/target/fda78e1e/resetallattrs_2.12-1.0.0.jar:/tmp/sbt_dc4cf587/target/92c158af/jersey-server-2.27.jar:/tmp/sbt_dc4cf587/target/9ca7d55d/netty-3.10.5.Final.jar:/tmp/sbt_dc4cf587/target/8531ad5a/jaxb-api-2.3.1.jar:/tmp/sbt_dc4cf587/target/bbce493f/scala-parser-combinators_2.12-1.1.0.jar:/tmp/sbt_dc4cf587/target/e3016167/jackson-module-paranamer-2.9.5.jar:/tm!
p/sbt_dc4cf587/target/9e0c05fa/akka-stream-alpakka-s3_2.12-0.14.jar:/tmp/sbt_dc4cf587/target/6d25f55a/jackson-module-scala_2.12-2.9.5.jar:/tmp/sbt_dc4cf587/target/7d73a975/play-json_2.12-2.6.7.jar:/tmp/sbt_dc4cf587/target/ebd5d04f/jersey-media-multipart-2.27.jar:/tmp/sbt_dc4cf587/target/9e8c4db1/akka-actor_2.12-2.5.14.jar:/tmp/sbt_dc4cf587/target/7fd4e3d0/hk2-api-2.5.0-b42.jar:/tmp/sbt_dc4cf587/target/bbbb9b5d/jetty-servlet-9.4.8.v20171121.jar:/tmp/sbt_dc4cf587/target/b31526a2/httpcore-4.4.4.jar:/tmp/sbt_dc4cf587/target/6975da39/javax.inject-1.jar:/tmp/sbt_dc4cf587/target/a7976caa/raven-8.0.3.jar:/tmp/sbt_dc4cf587/target/81e24b17/jersey-container-servlet-core-2.27.jar:/tmp/sbt_dc4cf587/target/89507701/guava-20.0.jar:/tmp/sbt_dc4cf587/target/0100b353/mesos-1.8.0.jar:/tmp/sbt_dc4cf587/target/106f2237/jersey-media-jaxb-2.27.jar:/tmp/sbt_dc4cf587/target/9e1ac84e/HdrHistogram-2.1.10.jar:/tmp/sbt_dc4cf587/target/15c5e958/commons-compress-1.13.jar:/tmp/sbt_dc4cf587/target/d3fe2dfa!
/jetty-io-9.4.8.v20171121.jar:/tmp/sbt_dc4cf587/target/66dd6a6!
2/jersey-client-2.27.jar:/tmp/sbt_dc4cf587/target/c845703d/commons-beanutils-1.9.3.jar:/tmp/sbt_dc4cf587/target/e0db9e57/aws-java-sdk-core-1.11.243.jar:/tmp/sbt_dc4cf587/target/8ad72fe3/commons-collections-3.2.2.jar:/tmp/sbt_dc4cf587/target/fbb86b57/scallop_2.12-3.1.2.jar:/tmp/sbt_dc4cf587/target/aa97ee9f/jersey-container-servlet-2.27.jar:/tmp/sbt_dc4cf587/target/7e256d16/play-functional_2.12-2.6.7.jar:/tmp/sbt_dc4cf587/target/2f22c882/jul-to-slf4j-1.7.21.jar:/tmp/sbt_dc4cf587/target/4474ff89/metrics-healthchecks-4.0.2.jar:/tmp/sbt_dc4cf587/target/faddf054/metrics-jvm-4.0.2.jar:/tmp/sbt_dc4cf587/target/a41fe6c2/akka-http_2.12-10.0.11.jar:/tmp/sbt_dc4cf587/target/323964c3/reactive-streams-1.0.2.jar:/tmp/sbt_dc4cf587/target/99a18e9a/jline-0.9.94.jar:/tmp/sbt_dc4cf587/target/34614bd9/jetty-server-9.4.8.v20171121.jar:/tmp/sbt_dc4cf587/target/7c4f3c47/logback-classic-1.2.3.jar:/tmp/sbt_dc4cf587/target/18c09ad0/scala-library.jar:/tmp/sbt_dc4cf587/target/813b4007/scala-async_2.12-!
0.9.7.jar:/tmp/sbt_dc4cf587/target/e22de336/scala-xml_2.12-1.0.6.jar:/tmp/sbt_dc4cf587/target/a22ac510/jackson-core-2.9.5.jar:/tmp/sbt_dc4cf587/target/4b68c2d5/config-1.3.3.jar:/tmp/sbt_dc4cf587/target/9ea08f93/jersey-hk2-2.27.jar:/tmp/sbt_dc4cf587/target/0ffc2cd3/jackson-datatype-jdk8-2.8.9.jar:/tmp/sbt_dc4cf587/target/c5f3355a/curator-x-async-4.0.1.jar:/tmp/sbt_dc4cf587/target/a6982b28/metrics-jetty9-4.0.2.jar:/tmp/sbt_dc4cf587/target/815893df/commons-io-2.6.jar:/tmp/sbt_dc4cf587/target/9525fb6b/scala-java8-compat_2.12-0.9.0.jar:/tmp/sbt_dc4cf587/target/2973d150/junit-4.12.jar:/tmp/sbt_dc4cf587/target/d6ec1a16/jetty-util-9.4.8.v20171121.jar:/tmp/sbt_dc4cf587/target/149700de/akka-parsing_2.12-10.0.11.jar:/tmp/sbt_dc4cf587/target/e74beab6/aopalliance-repackaged-2.5.0-b42.jar:/tmp/sbt_dc4cf587/target/34b64138/jetty-continuation-9.4.8.v20171121.jar:/tmp/sbt_dc4cf587/target/f7b7f3d6/jetty-servlets-9.4.8.v20171121.jar:/tmp/sbt_dc4cf587/target/9802a9a8/akka-http-play-json_2.12-1!
.18.1.jar:/tmp/sbt_dc4cf587/target/9954f282/raven-logback-8.0.3.jar:/tm!
p/sbt_dc4cf587/target/41c913d7/mimepull-1.9.6.jar:/tmp/sbt_dc4cf587/target/138093a4/profiler-1.0.2.jar:/tmp/sbt_dc4cf587/target/44eaf099/javassist-3.22.0-CR2.jar:/tmp/sbt_dc4cf587/target/981f7696/akka-stream_2.12-2.5.14.jar:/tmp/sbt_dc4cf587/target/5613f735/akka-http-xml_2.12-10.0.9.jar:/tmp/sbt_dc4cf587/target/f20d4304/jersey-common-2.27.jar:/tmp/sbt_dc4cf587/target/a4833298/ui-1.3.1.jar:/tmp/sbt_dc4cf587/target/619eba74/paranamer-2.8.jar:/tmp/sbt_dc4cf587/target/ae83b2b7/java-uuid-generator-3.1.4.jar:/tmp/sbt_dc4cf587/target/85262acf/javax.activation-api-1.2.0.jar:/tmp/sbt_dc4cf587/target/ec987884/metrics-core-4.0.2.jar:/tmp/sbt_dc4cf587/target/ee9dacea/ion-java-1.0.2.jar:/tmp/sbt_dc4cf587/target/42a25dc3/hamcrest-core-1.3.jar:/tmp/sbt_dc4cf587/target/f7b520c4/joda-time-2.9.9.jar:/tmp/sbt_dc4cf587/target/8d6f47e7/curator-recipes-4.0.1.jar:/tmp/sbt_dc4cf587/target/060f4378/jackson-datatype-jsr310-2.8.9.jar:/tmp/sbt_dc4cf587/target/b0debcff/akka-stream-alpakka-simple-codecs!
_2.12-0.14.jar:/tmp/sbt_dc4cf587/target/9ec90e5c/akka-http-core_2.12-10.0.11.jar:/tmp/sbt_dc4cf587/target/3da85d2b/curator-framework-4.0.1.jar:/tmp/sbt_dc4cf587/target/eb02b345/metrics-servlets-4.0.2.jar:/tmp/sbt_dc4cf587/target/426a0862/javax.ws.rs-api-2.1.jar:/tmp/sbt_dc4cf587/target/1b74e211/metrics-annotation-4.0.2.jar:/tmp/sbt_dc4cf587/target/8613ae82/validation-api-1.1.0.Final.jar:/tmp/sbt_dc4cf587/target/dc767a0b/zookeeper-3.4.11.jar:/tmp/sbt_dc4cf587/target/e8350eec/jetty-security-9.4.8.v20171121.jar:/tmp/sbt_dc4cf587/target/869c59c0/jgrapht-core-0.9.3.jar:/tmp/sbt_dc4cf587/target/38269f29/jackson-jaxrs-base-2.9.5.jar:/tmp/sbt_dc4cf587/target/f49e7919/accord-api_2.12-0.7.1.jar:/tmp/sbt_dc4cf587/target/da76ca59/slf4j-api-1.7.25.jar:/tmp/sbt_dc4cf587/target/ba9e74b1/jackson-dataformat-cbor-2.6.7.jar:/tmp/sbt_dc4cf587/target/a1dc9750/scala-logging_2.12-3.7.2.jar:/tmp/sbt_dc4cf587/target/ed809d26/macro-compat_2.12-1.1.1.jar:/tmp/sbt_dc4cf587/target/7440a22c/api-console-!
3.0.8-accept.jar:/tmp/sbt_dc4cf587/target/9d6edc0c/hk2-locator-2.5.0-b4!
2.jar:/tmp/sbt_dc4cf587/target/0235ba8b/aopalliance-1.0.jar:/tmp/sbt_dc4cf587/target/34905083/jackson-databind-2.9.5.jar:/tmp/sbt_dc4cf587/target/86434440/logback-core-1.2.3.jar:/tmp/sbt_dc4cf587/target/7eefbf15/accord-core_2.12-0.7.1.jar:/tmp/sbt_dc4cf587/target/98e0b7dc/javax.inject-2.5.0-b42.jar:/tmp/sbt_dc4cf587/target/b42ebb11/metrics-jersey2-4.0.2.jar:/tmp/sbt_dc4cf587/target/e2e76f0f/curator-client-4.0.1.jar:/tmp/sbt_dc4cf587/target/102a344e/hk2-utils-2.5.0-b42.jar:/tmp/sbt_dc4cf587/target/ab9443bc/metrics-json-4.0.2.jar:/tmp/sbt_dc4cf587/target/934c04d3/javax.annotation-api-1.3.2.jar:/tmp/sbt_dc4cf587/target/9056ec9d/jackson-annotations-2.9.5.jar:/tmp/sbt_dc4cf587/target/057fe178/rolling-metrics-2.0.4.jar:/tmp/sbt_dc4cf587/target/38b14ff1/akka-slf4j_2.12-2.5.14.jar:/tmp/sbt_dc4cf587/target/200fb936/protobuf-java-3.5.0.jar:/tmp/sbt_dc4cf587/target/4ed2b2d4/osgi-resource-locator-1.0.1.jar:/tmp/sbt_dc4cf587/target/c80c1d72/jackson-jaxrs-json-provider-2.9.5.jar:/tmp/sbt!
_dc4cf587/target/4d35b3e5/akka-protobuf_2.12-2.5.14.jar:/tmp/sbt_dc4cf587/target/c1fdf6c0/ssl-config-core_2.12-0.2.3.jar:/tmp/sbt_dc4cf587/target/733db77a/httpclient-4.5.2.jar:/tmp/sbt_dc4cf587/target/3cd63d07/javax.servlet-api-3.1.0.jar:/tmp/sbt_dc4cf587/target/eeb69005/guice-4.1.0.jar (org.apache.zookeeper.ZooKeeper:main)
2020-07-18 15:23:30,924] INFO Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper:main)
2020-07-18 15:23:30,928] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper:main)
2020-07-18 15:23:30,928] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper:main)
2020-07-18 15:23:30,929] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper:main)
2020-07-18 15:23:30,929] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper:main)
2020-07-18 15:23:30,929] INFO Client environment:os.version=3.10.0-1127.10.1.el7.x86_64 (org.apache.zookeeper.ZooKeeper:main)
2020-07-18 15:23:30,929] INFO Client environment:user.name=marc (org.apache.zookeeper.ZooKeeper:main)
2020-07-18 15:23:30,929] INFO Client environment:user.home=/home/users/marc (org.apache.zookeeper.ZooKeeper:main)
2020-07-18 15:23:30,929] INFO Client environment:user.dir=/home/software/marathon2 (org.apache.zookeeper.ZooKeeper:main)
2020-07-18 15:23:30,936] INFO Initiating client connection, connectString=m01.local:2181 sessionTimeout=10000 watcher=org.apache.curator.ConnectionState@6f89f665 (org.apache.zookeeper.ZooKeeper:main)
2020-07-18 15:23:30,950] DEBUG zookeeper.disableAutoWatchReset is false (org.apache.zookeeper.ClientCnxn:main)
2020-07-18 15:23:31,037] INFO Default schema (org.apache.curator.framework.imps.CuratorFrameworkImpl:main)
2020-07-18 15:23:31,046] INFO Opening socket connection to server m01.local/192.168.10.151:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:31,064] INFO Socket connection established to m01.local/192.168.10.151:2181, initiating session (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:31,083] DEBUG Session establishment request sent on m01.local/192.168.10.151:2181 (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:31,104] TRACE readConnectResult 37 0x[0,0,0,0,0,0,27,10,1,73,61,ffffffe5,ffffffba,70,0,d,0,0,0,10,5a,2f,ffffff94,ffffff9a,67,2b,19,34,ffffff84,ffffffde,28,ffffffae,27,ffffffba,20,1f,0,] (org.apache.zookeeper.ClientCnxnSocket:main-SendThread(m01.local:2181))
2020-07-18 15:23:31,108] INFO Session establishment complete on server m01.local/192.168.10.151:2181, sessionid = 0x17361e5ba70000d, negotiated timeout = 10000 (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:31,118] TRACE Counter SyncConnected: 1 (org.apache.curator.utils.DefaultTracerDriver:main-EventThread)
2020-07-18 15:23:31,121] DEBUG Negotiated session timeout: 10000 (org.apache.curator.ConnectionState:main-EventThread)
2020-07-18 15:23:31,129] INFO State change: CONNECTED (org.apache.curator.framework.state.ConnectionStateManager:main-EventThread)
2020-07-18 15:23:31,145] DEBUG Clearing sleep for 0 operations (org.apache.curator.framework.imps.CuratorFrameworkImpl:main-EventThread)
2020-07-18 15:23:31,148] TRACE Trace: connection-state-parent-process - 22 ms (org.apache.curator.utils.DefaultTracerDriver:main-EventThread)
2020-07-18 15:23:31,223] DEBUG Reading reply sessionid:0x17361e5ba70000d, packet:: clientPath:null serverPath:null finished:false header:: 1,3 replyHeader:: 1,103,0 request:: '/marathon,F response:: s{12,12,1595075395559,1595075395559,0,2,0,0,0,2,16} (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:31,230] DEBUG Reading reply sessionid:0x17361e5ba70000d, packet:: clientPath:null serverPath:null finished:false header:: 2,3 replyHeader:: 2,103,0 request:: '/marathon/state,F response:: s{13,13,1595075395583,1595075395583,0,11,0,0,0,7,93} (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:31,236] DEBUG Reading reply sessionid:0x17361e5ba70000d, packet:: clientPath:/marathon/state serverPath:/marathon/state finished:false header:: 3,9 replyHeader:: 3,103,0 request:: '/marathon/state response:: '/marathon/state (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:31,243] TRACE Trace: SyncBuilderImpl-Background - 85 ms (org.apache.curator.utils.DefaultTracerDriver:main-EventThread)
2020-07-18 15:23:32,307] INFO Started TaskTrackerUpdateStepsProcessorImpl with steps: [info] * continueOnError(notifyHealthCheckManager)
[info] * continueOnError(notifyRateLimiter)
[info] * continueOnError(notifyLaunchQueue)
[info] * continueOnError(postTaskStatusEvent)
[info] * continueOnError(scaleApp) (mesosphere.marathon.core.task.tracker.impl.InstanceTrackerUpdateStepProcessorImpl:main)
2020-07-18 15:23:32,867] INFO Calling reviveOffers is enabled. Use --disable_revive_offers_for_new_apps to disable. (mesosphere.marathon.core.flow.FlowModule:main)
2020-07-18 15:23:33,173] INFO Loading plugins implementing 'mesosphere.marathon.plugin.task.RunSpecTaskProcessor' from these urls: [] (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
2020-07-18 15:23:33,239] INFO Found 0 plugins. (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
2020-07-18 15:23:33,242] INFO Loading plugins implementing 'mesosphere.marathon.plugin.validation.RunSpecValidator' from these urls: [] (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
2020-07-18 15:23:33,250] INFO Found 0 plugins. (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
2020-07-18 15:23:33,342] INFO Loading plugins implementing 'mesosphere.marathon.plugin.auth.Authenticator' from these urls: [] (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
2020-07-18 15:23:33,344] INFO Found 0 plugins. (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
2020-07-18 15:23:33,355] INFO Loading plugins implementing 'mesosphere.marathon.plugin.auth.Authorizer' from these urls: [] (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
2020-07-18 15:23:33,360] INFO Found 0 plugins. (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
2020-07-18 15:23:33,442] INFO Started status update processor (mesosphere.marathon.core.task.update.impl.TaskStatusUpdateProcessorImpl:main)
2020-07-18 15:23:33,502] INFO Loading plugins implementing 'mesosphere.marathon.plugin.scheduler.SchedulerPlugin' from these urls: [] (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
2020-07-18 15:23:33,504] INFO Found 0 plugins. (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
2020-07-18 15:23:33,684] INFO All actors suspended: [info] * Actor[akka://marathon/user/reviveOffersWhenWanted#131319476]
[info] * Actor[akka://marathon/user/deploymentManager#1520374777]
[info] * Actor[akka://marathon/user/taskKillServiceActor#-1807863507]
[info] * Actor[akka://marathon/user/offersWantedForReconciliation#-736352801]
[info] * Actor[akka://marathon/user/rateLimiter#-1296343297]
[info] * Actor[akka://marathon/user/instanceTracker#-1145393542]
[info] * Actor[akka://marathon/user/killOverdueStagedTasks#-1014414263]
[info] * Actor[akka://marathon/user/offerMatcherLaunchTokens#1705741668]
[info] * Actor[akka://marathon/user/offerMatcherManager#-1248363557]
[info] * Actor[akka://marathon/user/expungeOverdueLostTasks#1659678442]
[info] * Actor[akka://marathon/user/launchQueue#-1257179790] (mesosphere.marathon.core.leadership.impl.LeadershipCoordinatorActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:33,710] INFO Created Mesos heartbeat monitor for scheduler mesosphere.marathon.MarathonScheduler@9e2ad91 (mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor:main)
2020-07-18 15:23:33,712] DEBUG using scheduler mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor@47a7a101 (mesosphere.marathon.MesosSchedulerDriverFactory:main)
2020-07-18 15:23:33,745] INFO Created Mesos heartbeat monitor for scheduler mesosphere.marathon.MarathonScheduler@9e2ad91 (mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor:main)
2020-07-18 15:23:33,782] INFO History actor ready (mesosphere.marathon.core.history.impl.HistoryActor:marathon-akka.actor.default-dispatcher-2)
2020-07-18 15:23:33,834] DEBUG Logging to Logger[org.eclipse.jetty.util.log] via org.eclipse.jetty.util.log.Slf4jLog (org.eclipse.jetty.util.log:main)
2020-07-18 15:23:33,844] INFO Logging initialized @10341ms to org.eclipse.jetty.util.log.Slf4jLog (org.eclipse.jetty.util.log:main)
2020-07-18 15:23:33,864] DEBUG org.eclipse.jetty.server.Server@140d1230[9.4.8.v20171121] added {QueuedThreadPool@qtp69670055{STOPPED,8<=0<=200,i=0,q=0},AUTO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:33,917] DEBUG HttpConnectionFactory@61c42416[HTTP/1.1] added {HttpConfiguration@329a1f8d{32768/8192,8192/8192,https://:8443,[]},POJO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:33,936] DEBUG ServerConnector@3a175162{null,[]}{0.0.0.0:0} added {org.eclipse.jetty.server.Server@140d1230[9.4.8.v20171121],UNMANAGED} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:33,946] DEBUG ServerConnector@3a175162{null,[]}{0.0.0.0:0} added {QueuedThreadPool@qtp69670055{STOPPED,8<=0<=200,i=0,q=0},AUTO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:33,948] DEBUG ServerConnector@3a175162{null,[]}{0.0.0.0:0} added {org.eclipse.jetty.util.thread.ScheduledExecutorScheduler@7569ea63,AUTO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:33,949] DEBUG ServerConnector@3a175162{null,[]}{0.0.0.0:0} added {org.eclipse.jetty.io.ArrayByteBufferPool@1640190a,POJO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:33,953] DEBUG ServerConnector@3a175162{null,[http/1.1]}{0.0.0.0:0} added {HttpConnectionFactory@61c42416[HTTP/1.1],AUTO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:33,955] DEBUG ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:0} added HttpConnectionFactory@61c42416[HTTP/1.1] (org.eclipse.jetty.server.AbstractConnector:main)
2020-07-18 15:23:33,963] DEBUG ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:0} added {SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:0},MANAGED} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:33,968] INFO Adding HTTP support. (mesosphere.marathon.api.HttpModule:main)
2020-07-18 15:23:33,970] DEBUG org.eclipse.jetty.server.Server@140d1230[9.4.8.v20171121] added {ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080},AUTO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:33,972] INFO No HTTPS support configured. (mesosphere.marathon.api.HttpModule:main)
2020-07-18 15:23:34,002] DEBUG HttpField encoders loaded: [org.eclipse.jetty.http.Http1FieldPreEncoder@475646d4] (org.eclipse.jetty.http.PreEncodedHttpField:main)
2020-07-18 15:23:34,048] DEBUG Added MappedResource[pathSpec=ServletPathSpec[".svgz",pathDepth=0,group=SUFFIX_GLOB],resource=true] to PathMappings[size=1] (org.eclipse.jetty.http.pathmap.PathMappings:main)
2020-07-18 15:23:34,055] DEBUG org.eclipse.jetty.server.handler.gzip.GzipHandler@7dd611c8 mime types IncludeExclude@5d425813{i=[],ip=org.eclipse.jetty.util.IncludeExcludeSet$SetContainsPredicate@1702830d,e=[image/ief, image/vnd.wap.wbmp, image/jpeg, application/bzip2, image/x-portable-graymap, application/brotli, image/bmp, image/gif, image/x-icon, audio/midi, video/x-msvideo, image/x-xbitmap, application/x-rar-compressed, image/x-portable-bitmap, image/x-rgb, image/x-cmu-raster, application/gzip, audio/x-wav, audio/x-pn-realaudio, audio/basic, application/compress, audio/x-aiff, video/x.ms.asx, video/x.ms.asf, image/png, video/vnd.rn-realvideo, image/x-xwindowdump, image/jpeg2000, video/x-sgi-movie, audio/mpeg, image/xcf, video/mpeg, image/x-portable-pixmap, image/tiff, image/x-portable-anymap, image/x-xpixmap, application/zip, video/quicktime, application/x-xz, video/mp4],ep=org.eclipse.jetty.util.IncludeExcludeSet$SetContainsPredicate@6a937!
336} (org.eclipse.jetty.server.handler.gzip.GzipHandler:main)
2020-07-18 15:23:34,092] DEBUG Adding Decorator: org.eclipse.jetty.util.DeprecationWarning@56afdf9a (org.eclipse.jetty.util.DecoratedObjectFactory:main)
2020-07-18 15:23:34,103] DEBUG o.e.j.s.ServletContextHandler@67efd2c2{/,null,UNAVAILABLE} added {org.eclipse.jetty.servlet.ServletHandler@71d9cb05,MANAGED} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:34,111] DEBUG mesosphere.marathon.metrics.current.InstrumentedHandler@665522c2 added {o.e.j.s.ServletContextHandler@67efd2c2{/,null,UNAVAILABLE},MANAGED} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:34,116] DEBUG org.eclipse.jetty.server.handler.HandlerCollection@2016f509[mesosphere.marathon.metrics.current.InstrumentedHandler@665522c2] added {mesosphere.marathon.metrics.current.InstrumentedHandler@665522c2,AUTO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:34,121] DEBUG org.eclipse.jetty.server.handler.HandlerCollection@2016f509[mesosphere.marathon.metrics.current.InstrumentedHandler@665522c2, mesosphere.marathon.api.HttpTransferMetricsHandler@78c1372d] added {mesosphere.marathon.api.HttpTransferMetricsHandler@78c1372d,AUTO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:34,138] DEBUG org.eclipse.jetty.server.handler.RequestLogHandler@3c69362a added {mesosphere.marathon.api.JettyRequestLog@2412a42b,AUTO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:34,139] DEBUG org.eclipse.jetty.server.handler.HandlerCollection@2016f509[mesosphere.marathon.metrics.current.InstrumentedHandler@665522c2, mesosphere.marathon.api.HttpTransferMetricsHandler@78c1372d, org.eclipse.jetty.server.handler.RequestLogHandler@3c69362a] added {org.eclipse.jetty.server.handler.RequestLogHandler@3c69362a,AUTO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:34,140] DEBUG org.eclipse.jetty.server.handler.gzip.GzipHandler@7dd611c8 added {org.eclipse.jetty.server.handler.HandlerCollection@2016f509[mesosphere.marathon.metrics.current.InstrumentedHandler@665522c2, mesosphere.marathon.api.HttpTransferMetricsHandler@78c1372d, org.eclipse.jetty.server.handler.RequestLogHandler@3c69362a],MANAGED} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:34,141] DEBUG org.eclipse.jetty.server.Server@140d1230[9.4.8.v20171121] added {org.eclipse.jetty.server.handler.gzip.GzipHandler@7dd611c8,MANAGED} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:34,147] INFO Starting Marathon 1.8.0-SNAPSHOT/unknown with --master m01.local:5050 --zk zk://m01.local:2181/marathon --framework_name marathon --http_port 8080 --logging_level trace --hostname test2.local (mesosphere.marathon.MarathonApp:main)
2020-07-18 15:23:34,363] INFO Loading plugins implementing 'mesosphere.marathon.plugin.http.HttpRequestHandler' from these urls: [] (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
2020-07-18 15:23:34,369] INFO Found 0 plugins. (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
2020-07-18 15:23:34,583] DEBUG Got ping response for sessionid: 0x17361e5ba70000d after 1ms (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:35,886] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {mesosphere.marathon.api.LeaderProxyFilter-5853ca50,AUTO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:35,890] INFO Now standing by. Closing existing handles and rejecting new. (mesosphere.marathon.core.event.impl.stream.HttpEventStreamActor:marathon-akka.actor.default-dispatcher-5)
2020-07-18 15:23:35,901] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {[/]/[]==31=>mesosphere.marathon.api.LeaderProxyFilter-5853ca50,POJO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:35,906] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {mesosphere.marathon.api.LimitConcurrentRequestsFilter-4a37191a,AUTO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:35,911] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {[/]/[]==31=>mesosphere.marathon.api.LimitConcurrentRequestsFilter-4a37191a,POJO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:35,913] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {mesosphere.marathon.api.CORSFilter-5854a18,AUTO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:35,914] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {[/]/[]==31=>mesosphere.marathon.api.CORSFilter-5854a18,POJO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:35,915] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {mesosphere.marathon.api.CacheDisablingFilter-d5556bf,AUTO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:35,916] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {[/]/[]==31=>mesosphere.marathon.api.CacheDisablingFilter-d5556bf,POJO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:35,919] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {mesosphere.marathon.core.event.impl.stream.HttpEventStreamServlet-67cefd84@24dd899f==mesosphere.marathon.core.event.impl.stream.HttpEventStreamServlet,jsp=null,order=-1,inst=true,AUTO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:35,929] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {[/v2/events]=>mesosphere.marathon.core.event.impl.stream.HttpEventStreamServlet-67cefd84,POJO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:35,931] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {mesosphere.marathon.api.WebJarServlet-12bbfc54@aafe7aab==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true,AUTO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:35,933] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {[/ui]=>mesosphere.marathon.api.WebJarServlet-12bbfc54,POJO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:35,934] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {mesosphere.marathon.api.WebJarServlet-1491344a@cc09e0b7==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true,AUTO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:35,937] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {[/ui/]=>mesosphere.marathon.api.WebJarServlet-1491344a,POJO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:35,939] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {mesosphere.marathon.api.WebJarServlet-59b65dce@ba4c8617==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true,AUTO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:35,940] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {[/help]=>mesosphere.marathon.api.WebJarServlet-59b65dce,POJO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:35,945] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {mesosphere.marathon.api.WebJarServlet-1386313f@95b536ff==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true,AUTO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:35,945] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {[/api-console]=>mesosphere.marathon.api.WebJarServlet-1386313f,POJO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:35,946] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {mesosphere.marathon.api.WebJarServlet-5e922647@8bad961e==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true,AUTO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:35,947] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {[/api-console/]=>mesosphere.marathon.api.WebJarServlet-5e922647,POJO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:35,949] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {mesosphere.marathon.api.PublicServlet-433c6abb@9e70036a==mesosphere.marathon.api.PublicServlet,jsp=null,order=-1,inst=true,AUTO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:35,951] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {[/public/]=>mesosphere.marathon.api.PublicServlet-433c6abb,POJO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:36,014] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {org.glassfish.jersey.servlet.ServletContainer-4eb166a1@7b93d9be==org.glassfish.jersey.servlet.ServletContainer,jsp=null,order=-1,inst=true,AUTO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:36,019] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {[/]=>org.glassfish.jersey.servlet.ServletContainer-4eb166a1,POJO} (org.eclipse.jetty.util.component.ContainerLifeCycle:main)
2020-07-18 15:23:36,070] INFO Starting up HttpServer. (mesosphere.marathon.api.MarathonHttpService:MarathonHttpService STARTING) 2020-07-18 15:23:36,072] DEBUG starting org.eclipse.jetty.server.Server@140d1230[9.4.8.v20171121] (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,078] INFO Starting up (mesosphere.marathon.MarathonSchedulerService:MarathonSchedulerService)
2020-07-18 15:23:36,080] INFO Beginning run (mesosphere.marathon.MarathonSchedulerService:MarathonSchedulerService)
2020-07-18 15:23:36,087] DEBUG org.eclipse.jetty.server.Server@140d1230[9.4.8.v20171121] added {org.eclipse.jetty.server.handler.ErrorHandler@449e0ef4,AUTO} (org.eclipse.jetty.util.component.ContainerLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,096] INFO jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8 (org.eclipse.jetty.server.Server:MarathonHttpService STARTING) 2020-07-18 15:23:36,148] DEBUG starting org.eclipse.jetty.server.Server@140d1230[9.4.8.v20171121] (org.eclipse.jetty.server.handler.AbstractHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,149] DEBUG starting QueuedThreadPool@qtp69670055{STOPPED,8<=0<=200,i=0,q=0} (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,161] INFO Starting leader latch (mesosphere.marathon.core.election.CuratorElectionStream$:MarathonSchedulerService)
2020-07-18 15:23:36,176] DEBUG STARTED @12674ms QueuedThreadPool@qtp69670055{STARTED,8<=8<=200,i=7,q=0} (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,176] DEBUG starting org.eclipse.jetty.server.handler.gzip.GzipHandler@7dd611c8 (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,186] DEBUG starting org.eclipse.jetty.server.handler.gzip.GzipHandler@7dd611c8 (org.eclipse.jetty.server.handler.AbstractHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,187] DEBUG starting org.eclipse.jetty.server.handler.HandlerCollection@2016f509[mesosphere.marathon.metrics.current.InstrumentedHandler@665522c2, mesosphere.marathon.api.HttpTransferMetricsHandler@78c1372d, org.eclipse.jetty.server.handler.RequestLogHandler@3c69362a] (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,188] DEBUG starting org.eclipse.jetty.server.handler.HandlerCollection@2016f509[mesosphere.marathon.metrics.current.InstrumentedHandler@665522c2, mesosphere.marathon.api.HttpTransferMetricsHandler@78c1372d, org.eclipse.jetty.server.handler.RequestLogHandler@3c69362a] (org.eclipse.jetty.server.handler.AbstractHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,192] DEBUG starting mesosphere.marathon.metrics.current.InstrumentedHandler@665522c2 (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,192] DEBUG starting mesosphere.marathon.metrics.current.InstrumentedHandler@665522c2 (org.eclipse.jetty.server.handler.AbstractHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,193] DEBUG starting o.e.j.s.ServletContextHandler@67efd2c2{/,null,UNAVAILABLE} (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,197] DEBUG starting o.e.j.s.ServletContextHandler@67efd2c2{/,null,STARTING} (org.eclipse.jetty.server.handler.AbstractHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,198] DEBUG starting org.eclipse.jetty.servlet.ServletHandler@71d9cb05 (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,203] DEBUG Path=/v2/events[EMBEDDED:null] mapped to servlet=mesosphere.marathon.core.event.impl.stream.HttpEventStreamServlet-67cefd84[EMBEDDED:null] (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,216] DEBUG Added MappedResource[pathSpec=ServletPathSpec["/v2/events",pathDepth=2,group=EXACT],resource=mesosphere.marathon.core.event.impl.stream.HttpEventStreamServlet-67cefd84@24dd899f==mesosphere.marathon.core.event.impl.stream.HttpEventStreamServlet,jsp=null,order=-1,inst=true] to PathMappings[size=1] (org.eclipse.jetty.http.pathmap.PathMappings:MarathonHttpService STARTING) 2020-07-18 15:23:36,217] DEBUG Path=/ui[EMBEDDED:null] mapped to servlet=mesosphere.marathon.api.WebJarServlet-12bbfc54[EMBEDDED:null] (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,218] DEBUG Added MappedResource[pathSpec=ServletPathSpec["/ui",pathDepth=1,group=EXACT],resource=mesosphere.marathon.api.WebJarServlet-12bbfc54@aafe7aab==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true] to PathMappings[size=2] (org.eclipse.jetty.http.pathmap.PathMappings:MarathonHttpService STARTING) 2020-07-18 15:23:36,219] DEBUG Path=/ui/[EMBEDDED:null] mapped to servlet=mesosphere.marathon.api.WebJarServlet-1491344a[EMBEDDED:null] (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,220] DEBUG Added MappedResource[pathSpec=ServletPathSpec["/ui/",pathDepth=2,group=PREFIX_GLOB],resource=mesosphere.marathon.api.WebJarServlet-1491344a@cc09e0b7==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true] to PathMappings[size=3] (org.eclipse.jetty.http.pathmap.PathMappings:MarathonHttpService STARTING) 2020-07-18 15:23:36,221] DEBUG Path=/api-console[EMBEDDED:null] mapped to servlet=mesosphere.marathon.api.WebJarServlet-1386313f[EMBEDDED:null] (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,222] DEBUG Added MappedResource[pathSpec=ServletPathSpec["/api-console",pathDepth=1,group=EXACT],resource=mesosphere.marathon.api.WebJarServlet-1386313f@95b536ff==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true] to PathMappings[size=4] (org.eclipse.jetty.http.pathmap.PathMappings:MarathonHttpService STARTING) 2020-07-18 15:23:36,224] DEBUG Leader watch loop 1 registering watch (mesosphere.marathon.core.election.CuratorElectionStream$:pool-1-thread-1)
2020-07-18 15:23:36,226] DEBUG Path=/help[EMBEDDED:null] mapped to servlet=mesosphere.marathon.api.WebJarServlet-59b65dce[EMBEDDED:null] (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,227] DEBUG Added MappedResource[pathSpec=ServletPathSpec["/help",pathDepth=1,group=EXACT],resource=mesosphere.marathon.api.WebJarServlet-59b65dce@ba4c8617==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true] to PathMappings[size=5] (org.eclipse.jetty.http.pathmap.PathMappings:MarathonHttpService STARTING) 2020-07-18 15:23:36,229] DEBUG Path=/public/[EMBEDDED:null] mapped to servlet=mesosphere.marathon.api.PublicServlet-433c6abb[EMBEDDED:null] (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,230] DEBUG Added MappedResource[pathSpec=ServletPathSpec["/public/",pathDepth=2,group=PREFIX_GLOB],resource=mesosphere.marathon.api.PublicServlet-433c6abb@9e70036a==mesosphere.marathon.api.PublicServlet,jsp=null,order=-1,inst=true] to PathMappings[size=6] (org.eclipse.jetty.http.pathmap.PathMappings:MarathonHttpService STARTING) 2020-07-18 15:23:36,232] DEBUG Path=/[EMBEDDED:null] mapped to servlet=org.glassfish.jersey.servlet.ServletContainer-4eb166a1[EMBEDDED:null] (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,235] DEBUG Added MappedResource[pathSpec=ServletPathSpec["/",pathDepth=1,group=PREFIX_GLOB],resource=org.glassfish.jersey.servlet.ServletContainer-4eb166a1@7b93d9be==org.glassfish.jersey.servlet.ServletContainer,jsp=null,order=-1,inst=true] to PathMappings[size=7] (org.eclipse.jetty.http.pathmap.PathMappings:MarathonHttpService STARTING) 2020-07-18 15:23:36,236] DEBUG Path=/api-console/[EMBEDDED:null] mapped to servlet=mesosphere.marathon.api.WebJarServlet-5e922647[EMBEDDED:null] (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,237] DEBUG Added MappedResource[pathSpec=ServletPathSpec["/api-console/",pathDepth=2,group=PREFIX_GLOB],resource=mesosphere.marathon.api.WebJarServlet-5e922647@8bad961e==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true] to PathMappings[size=8] (org.eclipse.jetty.http.pathmap.PathMappings:MarathonHttpService STARTING) 2020-07-18 15:23:36,238] DEBUG filterNameMap={mesosphere.marathon.api.CacheDisablingFilter-d5556bf=mesosphere.marathon.api.CacheDisablingFilter-d5556bf, mesosphere.marathon.api.LimitConcurrentRequestsFilter-4a37191a=mesosphere.marathon.api.LimitConcurrentRequestsFilter-4a37191a, mesosphere.marathon.api.CORSFilter-5854a18=mesosphere.marathon.api.CORSFilter-5854a18, mesosphere.marathon.api.LeaderProxyFilter-5853ca50=mesosphere.marathon.api.LeaderProxyFilter-5853ca50} (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,240] DEBUG pathFilters=[[/]/[]==31=>mesosphere.marathon.api.LeaderProxyFilter-5853ca50, [/]/[]==31=>mesosphere.marathon.api.LimitConcurrentRequestsFilter-4a37191a, [/]/[]==31=>mesosphere.marathon.api.CORSFilter-5854a18, [/]/[]==31=>mesosphere.marathon.api.CacheDisablingFilter-d5556bf] (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,243] DEBUG servletFilterMap={} (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,244] DEBUG servletPathMap=PathMappings[size=8] (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,250] DEBUG servletNameMap={mesosphere.marathon.core.event.impl.stream.HttpEventStreamServlet-67cefd84=mesosphere.marathon.core.event.impl.stream.HttpEventStreamServlet-67cefd84@24dd899f==mesosphere.marathon.core.event.impl.stream.HttpEventStreamServlet,jsp=null,order=-1,inst=true, mesosphere.marathon.api.WebJarServlet-5e922647=mesosphere.marathon.api.WebJarServlet-5e922647@8bad961e==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true, mesosphere.marathon.api.WebJarServlet-12bbfc54=mesosphere.marathon.api.WebJarServlet-12bbfc54@aafe7aab==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true, mesosphere.marathon.api.WebJarServlet-1386313f=mesosphere.marathon.api.WebJarServlet-1386313f@95b536ff==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true, mesosphere.marathon.api.PublicServlet-433c6abb=mesosphere.marathon.api.PublicServlet-433c6abb@9e70036a==mesosphere.marathon.api.PublicServlet,!
jsp=null,order=-1,inst=true, mesosphere.marathon.api.WebJarServlet-59b65dce=mesosphere.marathon.api.WebJarServlet-59b65dce@ba4c8617==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true, org.glassfish.jersey.servlet.ServletContainer-4eb166a1=org.glassfish.jersey.servlet.ServletContainer-4eb166a1@7b93d9be==org.glassfish.jersey.servlet.ServletContainer,jsp=null,order=-1,inst=true, mesosphere.marathon.api.WebJarServlet-1491344a=mesosphere.marathon.api.WebJarServlet-1491344a@cc09e0b7==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true} (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,255] DEBUG Adding Default404Servlet to org.eclipse.jetty.servlet.ServletHandler@71d9cb05 (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,266] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {org.eclipse.jetty.servlet.ServletHandler$Default404Servlet-51f846dc@b0145943==org.eclipse.jetty.servlet.ServletHandler$Default404Servlet,jsp=null,order=-1,inst=false,AUTO} (org.eclipse.jetty.util.component.ContainerLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,267] DEBUG org.eclipse.jetty.servlet.ServletHandler@71d9cb05 added {[/]=>org.eclipse.jetty.servlet.ServletHandler$Default404Servlet-51f846dc,POJO} (org.eclipse.jetty.util.component.ContainerLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,268] DEBUG Path=/v2/events[EMBEDDED:null] mapped to servlet=mesosphere.marathon.core.event.impl.stream.HttpEventStreamServlet-67cefd84[EMBEDDED:null] (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,269] DEBUG Added MappedResource[pathSpec=ServletPathSpec["/v2/events",pathDepth=2,group=EXACT],resource=mesosphere.marathon.core.event.impl.stream.HttpEventStreamServlet-67cefd84@24dd899f==mesosphere.marathon.core.event.impl.stream.HttpEventStreamServlet,jsp=null,order=-1,inst=true] to PathMappings[size=1] (org.eclipse.jetty.http.pathmap.PathMappings:MarathonHttpService STARTING) 2020-07-18 15:23:36,271] DEBUG Path=/ui[EMBEDDED:null] mapped to servlet=mesosphere.marathon.api.WebJarServlet-12bbfc54[EMBEDDED:null] (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,272] DEBUG Added MappedResource[pathSpec=ServletPathSpec["/ui",pathDepth=1,group=EXACT],resource=mesosphere.marathon.api.WebJarServlet-12bbfc54@aafe7aab==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true] to PathMappings[size=2] (org.eclipse.jetty.http.pathmap.PathMappings:MarathonHttpService STARTING) 2020-07-18 15:23:36,273] DEBUG Path=/ui/[EMBEDDED:null] mapped to servlet=mesosphere.marathon.api.WebJarServlet-1491344a[EMBEDDED:null] (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,274] DEBUG Added MappedResource[pathSpec=ServletPathSpec["/ui/",pathDepth=2,group=PREFIX_GLOB],resource=mesosphere.marathon.api.WebJarServlet-1491344a@cc09e0b7==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true] to PathMappings[size=3] (org.eclipse.jetty.http.pathmap.PathMappings:MarathonHttpService STARTING) 2020-07-18 15:23:36,276] DEBUG Path=/api-console[EMBEDDED:null] mapped to servlet=mesosphere.marathon.api.WebJarServlet-1386313f[EMBEDDED:null] (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,279] DEBUG Added MappedResource[pathSpec=ServletPathSpec["/api-console",pathDepth=1,group=EXACT],resource=mesosphere.marathon.api.WebJarServlet-1386313f@95b536ff==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true] to PathMappings[size=4] (org.eclipse.jetty.http.pathmap.PathMappings:MarathonHttpService STARTING) 2020-07-18 15:23:36,281] DEBUG Reading reply sessionid:0x17361e5ba70000d, packet:: clientPath:null serverPath:null finished:false header:: 4,12 replyHeader:: 4,103,0 request:: '/marathon/leader-curator,T response:: v{},s{16,16,1595075400424,1595075400424,0,10,0,0,0,0,101} (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:36,283] TRACE Trace: GetChildrenBuilderImpl-Foreground - 22 ms (org.apache.curator.utils.DefaultTracerDriver:pool-1-thread-1)
2020-07-18 15:23:36,288] DEBUG Path=/help[EMBEDDED:null] mapped to servlet=mesosphere.marathon.api.WebJarServlet-59b65dce[EMBEDDED:null] (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,289] DEBUG Added MappedResource[pathSpec=ServletPathSpec["/help",pathDepth=1,group=EXACT],resource=mesosphere.marathon.api.WebJarServlet-59b65dce@ba4c8617==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true] to PathMappings[size=5] (org.eclipse.jetty.http.pathmap.PathMappings:MarathonHttpService STARTING) 2020-07-18 15:23:36,289] DEBUG Path=/public/[EMBEDDED:null] mapped to servlet=mesosphere.marathon.api.PublicServlet-433c6abb[EMBEDDED:null] (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,292] DEBUG Added MappedResource[pathSpec=ServletPathSpec["/public/",pathDepth=2,group=PREFIX_GLOB],resource=mesosphere.marathon.api.PublicServlet-433c6abb@9e70036a==mesosphere.marathon.api.PublicServlet,jsp=null,order=-1,inst=true] to PathMappings[size=6] (org.eclipse.jetty.http.pathmap.PathMappings:MarathonHttpService STARTING) 2020-07-18 15:23:36,294] DEBUG Path=/[EMBEDDED:null] mapped to servlet=org.glassfish.jersey.servlet.ServletContainer-4eb166a1[EMBEDDED:null] (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,294] DEBUG Added MappedResource[pathSpec=ServletPathSpec["/",pathDepth=1,group=PREFIX_GLOB],resource=org.glassfish.jersey.servlet.ServletContainer-4eb166a1@7b93d9be==org.glassfish.jersey.servlet.ServletContainer,jsp=null,order=-1,inst=true] to PathMappings[size=7] (org.eclipse.jetty.http.pathmap.PathMappings:MarathonHttpService STARTING) 2020-07-18 15:23:36,294] DEBUG Path=/api-console/[EMBEDDED:null] mapped to servlet=mesosphere.marathon.api.WebJarServlet-5e922647[EMBEDDED:null] (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,294] DEBUG Added MappedResource[pathSpec=ServletPathSpec["/api-console/",pathDepth=2,group=PREFIX_GLOB],resource=mesosphere.marathon.api.WebJarServlet-5e922647@8bad961e==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true] to PathMappings[size=8] (org.eclipse.jetty.http.pathmap.PathMappings:MarathonHttpService STARTING) 2020-07-18 15:23:36,294] DEBUG Path=/[EMBEDDED:null] mapped to servlet=org.eclipse.jetty.servlet.ServletHandler$Default404Servlet-51f846dc[EMBEDDED:null] (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,294] DEBUG Added MappedResource[pathSpec=ServletPathSpec["/",pathDepth=-1,group=DEFAULT],resource=org.eclipse.jetty.servlet.ServletHandler$Default404Servlet-51f846dc@b0145943==org.eclipse.jetty.servlet.ServletHandler$Default404Servlet,jsp=null,order=-1,inst=false] to PathMappings[size=9] (org.eclipse.jetty.http.pathmap.PathMappings:MarathonHttpService STARTING) 2020-07-18 15:23:36,295] DEBUG filterNameMap={mesosphere.marathon.api.CacheDisablingFilter-d5556bf=mesosphere.marathon.api.CacheDisablingFilter-d5556bf, mesosphere.marathon.api.LimitConcurrentRequestsFilter-4a37191a=mesosphere.marathon.api.LimitConcurrentRequestsFilter-4a37191a, mesosphere.marathon.api.CORSFilter-5854a18=mesosphere.marathon.api.CORSFilter-5854a18, mesosphere.marathon.api.LeaderProxyFilter-5853ca50=mesosphere.marathon.api.LeaderProxyFilter-5853ca50} (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,295] DEBUG pathFilters=[[/]/[]==31=>mesosphere.marathon.api.LeaderProxyFilter-5853ca50, [/]/[]==31=>mesosphere.marathon.api.LimitConcurrentRequestsFilter-4a37191a, [/]/[]==31=>mesosphere.marathon.api.CORSFilter-5854a18, [/]/[]==31=>mesosphere.marathon.api.CacheDisablingFilter-d5556bf] (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,295] DEBUG servletFilterMap={} (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,295] DEBUG servletPathMap=PathMappings[size=9] (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,295] DEBUG servletNameMap={mesosphere.marathon.core.event.impl.stream.HttpEventStreamServlet-67cefd84=mesosphere.marathon.core.event.impl.stream.HttpEventStreamServlet-67cefd84@24dd899f==mesosphere.marathon.core.event.impl.stream.HttpEventStreamServlet,jsp=null,order=-1,inst=true, mesosphere.marathon.api.WebJarServlet-5e922647=mesosphere.marathon.api.WebJarServlet-5e922647@8bad961e==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true, mesosphere.marathon.api.WebJarServlet-12bbfc54=mesosphere.marathon.api.WebJarServlet-12bbfc54@aafe7aab==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true, org.eclipse.jetty.servlet.ServletHandler$Default404Servlet-51f846dc=org.eclipse.jetty.servlet.ServletHandler$Default404Servlet-51f846dc@b0145943==org.eclipse.jetty.servlet.ServletHandler$Default404Servlet,jsp=null,order=-1,inst=false, mesosphere.marathon.api.WebJarServlet-1386313f=mesosphere.marathon.api.WebJarS!
ervlet-1386313f@95b536ff==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true, mesosphere.marathon.api.PublicServlet-433c6abb=mesosphere.marathon.api.PublicServlet-433c6abb@9e70036a==mesosphere.marathon.api.PublicServlet,jsp=null,order=-1,inst=true, mesosphere.marathon.api.WebJarServlet-59b65dce=mesosphere.marathon.api.WebJarServlet-59b65dce@ba4c8617==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true, org.glassfish.jersey.servlet.ServletContainer-4eb166a1=org.glassfish.jersey.servlet.ServletContainer-4eb166a1@7b93d9be==org.glassfish.jersey.servlet.ServletContainer,jsp=null,order=-1,inst=true, mesosphere.marathon.api.WebJarServlet-1491344a=mesosphere.marathon.api.WebJarServlet-1491344a@cc09e0b7==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true} (org.eclipse.jetty.servlet.ServletHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,295] DEBUG starting org.eclipse.jetty.servlet.ServletHandler@71d9cb05 (org.eclipse.jetty.server.handler.AbstractHandler:MarathonHttpService STARTING) 2020-07-18 15:23:36,295] DEBUG STARTED @12793ms org.eclipse.jetty.servlet.ServletHandler@71d9cb05 (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,320] DEBUG Reading reply sessionid:0x17361e5ba70000d, packet:: clientPath:null serverPath:null finished:false header:: 5,12 replyHeader:: 5,103,0 request:: '/marathon/leader-curator,F response:: v{},s{16,16,1595075400424,1595075400424,0,10,0,0,0,0,101} (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:36,322] TRACE Trace: GetChildrenBuilderImpl-Foreground - 3 ms (org.apache.curator.utils.DefaultTracerDriver:pool-1-thread-1)
2020-07-18 15:23:36,296] DEBUG starting mesosphere.marathon.api.LeaderProxyFilter-5853ca50 (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,324] DEBUG STARTED @12821ms mesosphere.marathon.api.LeaderProxyFilter-5853ca50 (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,325] DEBUG Filter.init mesosphere.marathon.api.LeaderProxyFilter@2f1565f8 (org.eclipse.jetty.servlet.FilterHolder:MarathonHttpService STARTING) 2020-07-18 15:23:36,325] DEBUG starting mesosphere.marathon.api.LimitConcurrentRequestsFilter-4a37191a (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,325] DEBUG STARTED @12823ms mesosphere.marathon.api.LimitConcurrentRequestsFilter-4a37191a (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,326] DEBUG Filter.init mesosphere.marathon.api.LimitConcurrentRequestsFilter@30f4d32c (org.eclipse.jetty.servlet.FilterHolder:MarathonHttpService STARTING) 2020-07-18 15:23:36,328] DEBUG starting mesosphere.marathon.api.CORSFilter-5854a18 (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,328] DEBUG STARTED @12826ms mesosphere.marathon.api.CORSFilter-5854a18 (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,336] DEBUG Filter.init mesosphere.marathon.api.CORSFilter@5ee6138c (org.eclipse.jetty.servlet.FilterHolder:MarathonHttpService STARTING) 2020-07-18 15:23:36,336] DEBUG starting mesosphere.marathon.api.CacheDisablingFilter-d5556bf (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,337] DEBUG STARTED @12834ms mesosphere.marathon.api.CacheDisablingFilter-d5556bf (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,337] DEBUG Filter.init mesosphere.marathon.api.CacheDisablingFilter@489c5fc0 (org.eclipse.jetty.servlet.FilterHolder:MarathonHttpService STARTING) 2020-07-18 15:23:36,337] DEBUG starting mesosphere.marathon.api.PublicServlet-433c6abb@9e70036a==mesosphere.marathon.api.PublicServlet,jsp=null,order=-1,inst=true (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,349] DEBUG STARTED @12847ms mesosphere.marathon.api.PublicServlet-433c6abb@9e70036a==mesosphere.marathon.api.PublicServlet,jsp=null,order=-1,inst=true (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,374] DEBUG Servlet.init mesosphere.marathon.api.PublicServlet@53b9e084 for mesosphere.marathon.api.PublicServlet-433c6abb (org.eclipse.jetty.servlet.ServletHolder:MarathonHttpService STARTING) 2020-07-18 15:23:36,414] DEBUG Leader watch loop 1: current leaders = (mesosphere.marathon.core.election.CuratorElectionStream$:pool-1-thread-1)
2020-07-18 15:23:36,414] DEBUG resource base = jar:file:/tmp/sbt_dc4cf587/job-1/target/9370c632/marathon_2.12-1.8.207.jar!/public (org.eclipse.jetty.servlet.DefaultServlet:MarathonHttpService STARTING) 2020-07-18 15:23:36,415] DEBUG starting mesosphere.marathon.api.WebJarServlet-12bbfc54@aafe7aab==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,415] DEBUG Got notification sessionid:0x17361e5ba70000d (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:36,418] DEBUG Got WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/marathon/leader-curator for sessionid 0x17361e5ba70000d (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:36,418] DEBUG Leader watch loop 1 event received: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/marathon/leader-curator (mesosphere.marathon.core.election.CuratorElectionStream$:main-EventThread)
2020-07-18 15:23:36,419] DEBUG Reading reply sessionid:0x17361e5ba70000d, packet:: clientPath:/marathon/leader-curator/_c_4ab60ec4-66e0-4701-bc00-f52a0325aa7c-latch- serverPath:/marathon/leader-curator/_c_4ab60ec4-66e0-4701-bc00-f52a0325aa7c-latch- finished:false header:: 6,1 replyHeader:: 6,104,0 request:: '/marathon/leader-curator/_c_4ab60ec4-66e0-4701-bc00-f52a0325aa7c-latch-,#74657374322e6c6f63616c3a38303830,v{s{31,s{'world,'anyone}}},3 response:: '/marathon/leader-curator/_c_4ab60ec4-66e0-4701-bc00-f52a0325aa7c-latch-0000000005 (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:36,422] TRACE Trace: CreateBuilderImpl-Background - 40 ms (org.apache.curator.utils.DefaultTracerDriver:main-EventThread)
2020-07-18 15:23:36,425] DEBUG Reading reply sessionid:0x17361e5ba70000d, packet:: clientPath:/marathon/leader-curator serverPath:/marathon/leader-curator finished:false header:: 7,12 replyHeader:: 7,104,0 request:: '/marathon/leader-curator,F response:: v{'_c_4ab60ec4-66e0-4701-bc00-f52a0325aa7c-latch-0000000005},s{16,16,1595075400424,1595075400424,0,11,0,0,0,1,104} (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:36,435] TRACE Trace: GetChildrenBuilderImpl-Background - 12 ms (org.apache.curator.utils.DefaultTracerDriver:main-EventThread)
2020-07-18 15:23:36,439] DEBUG STARTED @12912ms mesosphere.marathon.api.WebJarServlet-12bbfc54@aafe7aab==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,439] DEBUG Servlet.init mesosphere.marathon.api.WebJarServlet@6ba208ff for mesosphere.marathon.api.WebJarServlet-12bbfc54 (org.eclipse.jetty.servlet.ServletHolder:MarathonHttpService STARTING) 2020-07-18 15:23:36,440] DEBUG starting mesosphere.marathon.api.WebJarServlet-1386313f@95b536ff==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,444] DEBUG STARTED @12941ms mesosphere.marathon.api.WebJarServlet-1386313f@95b536ff==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,444] DEBUG Servlet.init mesosphere.marathon.api.WebJarServlet@6ba208ff for mesosphere.marathon.api.WebJarServlet-1386313f (org.eclipse.jetty.servlet.ServletHolder:MarathonHttpService STARTING) 2020-07-18 15:23:36,445] DEBUG starting mesosphere.marathon.api.WebJarServlet-1491344a@cc09e0b7==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,446] DEBUG STARTED @12943ms mesosphere.marathon.api.WebJarServlet-1491344a@cc09e0b7==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,447] DEBUG Servlet.init mesosphere.marathon.api.WebJarServlet@6ba208ff for mesosphere.marathon.api.WebJarServlet-1491344a (org.eclipse.jetty.servlet.ServletHolder:MarathonHttpService STARTING) 2020-07-18 15:23:36,447] DEBUG starting mesosphere.marathon.api.WebJarServlet-59b65dce@ba4c8617==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,449] DEBUG STARTED @12946ms mesosphere.marathon.api.WebJarServlet-59b65dce@ba4c8617==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,449] DEBUG Servlet.init mesosphere.marathon.api.WebJarServlet@6ba208ff for mesosphere.marathon.api.WebJarServlet-59b65dce (org.eclipse.jetty.servlet.ServletHolder:MarathonHttpService STARTING) 2020-07-18 15:23:36,450] DEBUG Leader watch loop 1 registering watch (mesosphere.marathon.core.election.CuratorElectionStream$:pool-1-thread-1)
2020-07-18 15:23:36,452] DEBUG starting mesosphere.marathon.api.WebJarServlet-5e922647@8bad961e==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,453] DEBUG STARTED @12950ms mesosphere.marathon.api.WebJarServlet-5e922647@8bad961e==mesosphere.marathon.api.WebJarServlet,jsp=null,order=-1,inst=true (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,454] DEBUG Servlet.init mesosphere.marathon.api.WebJarServlet@6ba208ff for mesosphere.marathon.api.WebJarServlet-5e922647 (org.eclipse.jetty.servlet.ServletHolder:MarathonHttpService STARTING) 2020-07-18 15:23:36,456] DEBUG Reading reply sessionid:0x17361e5ba70000d, packet:: clientPath:null serverPath:null finished:false header:: 8,12 replyHeader:: 8,104,0 request:: '/marathon/leader-curator,T response:: v{'_c_4ab60ec4-66e0-4701-bc00-f52a0325aa7c-latch-0000000005},s{16,16,1595075400424,1595075400424,0,11,0,0,0,1,104} (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:36,456] TRACE Trace: GetChildrenBuilderImpl-Foreground - 5 ms (org.apache.curator.utils.DefaultTracerDriver:pool-1-thread-1)
2020-07-18 15:23:36,461] DEBUG Reading reply sessionid:0x17361e5ba70000d, packet:: clientPath:null serverPath:null finished:false header:: 9,12 replyHeader:: 9,104,0 request:: '/marathon/leader-curator,F response:: v{'_c_4ab60ec4-66e0-4701-bc00-f52a0325aa7c-latch-0000000005},s{16,16,1595075400424,1595075400424,0,11,0,0,0,1,104} (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:36,464] DEBUG starting mesosphere.marathon.core.event.impl.stream.HttpEventStreamServlet-67cefd84@24dd899f==mesosphere.marathon.core.event.impl.stream.HttpEventStreamServlet,jsp=null,order=-1,inst=true (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,464] DEBUG STARTED @12961ms mesosphere.marathon.core.event.impl.stream.HttpEventStreamServlet-67cefd84@24dd899f==mesosphere.marathon.core.event.impl.stream.HttpEventStreamServlet,jsp=null,order=-1,inst=true (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,464] DEBUG Servlet.init mesosphere.marathon.core.event.impl.stream.HttpEventStreamServlet@7ffb8752 for mesosphere.marathon.core.event.impl.stream.HttpEventStreamServlet-67cefd84 (org.eclipse.jetty.servlet.ServletHolder:MarathonHttpService STARTING) 2020-07-18 15:23:36,464] DEBUG starting org.eclipse.jetty.servlet.ServletHandler$Default404Servlet-51f846dc@b0145943==org.eclipse.jetty.servlet.ServletHandler$Default404Servlet,jsp=null,order=-1,inst=false (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,464] DEBUG STARTED @12962ms org.eclipse.jetty.servlet.ServletHandler$Default404Servlet-51f846dc@b0145943==org.eclipse.jetty.servlet.ServletHandler$Default404Servlet,jsp=null,order=-1,inst=false (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,465] DEBUG starting org.glassfish.jersey.servlet.ServletContainer-4eb166a1@7b93d9be==org.glassfish.jersey.servlet.ServletContainer,jsp=null,order=-1,inst=true (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,468] TRACE Trace: GetChildrenBuilderImpl-Foreground - 12 ms (org.apache.curator.utils.DefaultTracerDriver:pool-1-thread-1)
2020-07-18 15:23:36,488] DEBUG STARTED @12985ms org.glassfish.jersey.servlet.ServletContainer-4eb166a1@7b93d9be==org.glassfish.jersey.servlet.ServletContainer,jsp=null,order=-1,inst=true (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:36,489] DEBUG Servlet.init org.glassfish.jersey.servlet.ServletContainer@3aaf799e for org.glassfish.jersey.servlet.ServletContainer-4eb166a1 (org.eclipse.jetty.servlet.ServletHolder:MarathonHttpService STARTING) 2020-07-18 15:23:36,535] DEBUG Reading reply sessionid:0x17361e5ba70000d, packet:: clientPath:null serverPath:null finished:false header:: 10,4 replyHeader:: 10,104,0 request:: '/marathon/leader-curator/_c_4ab60ec4-66e0-4701-bc00-f52a0325aa7c-latch-0000000005,F response:: #74657374322e6c6f63616c3a38303830,s{104,104,1595078616399,1595078616399,0,0,0,104534855662960653,16,0,104} (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:36,540] TRACE Trace: GetDataBuilderImpl-Foreground - 47 ms (org.apache.curator.utils.DefaultTracerDriver:pool-1-thread-1)
2020-07-18 15:23:36,540] DEBUG Leader watch loop 1: current leaders = test2.local:8080 (mesosphere.marathon.core.election.CuratorElectionStream$:pool-1-thread-1)
2020-07-18 15:23:36,543] INFO Leader won: test2.local:8080 (mesosphere.marathon.core.election.CuratorElectionStream$:marathon-akka.actor.default-dispatcher-5)
2020-07-18 15:23:36,550] INFO As new leader running the driver (mesosphere.marathon.MarathonSchedulerService:pool-1-thread-1)
2020-07-18 15:23:36,578] DEBUG Reading reply sessionid:0x17361e5ba70000d, packet:: clientPath:/marathon/state serverPath:/marathon/state finished:false header:: 11,9 replyHeader:: 11,104,0 request:: '/marathon/state response:: '/marathon/state (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:36,580] TRACE Trace: SyncBuilderImpl-Background - 4 ms (org.apache.curator.utils.DefaultTracerDriver:main-EventThread)
2020-07-18 15:23:36,652] DEBUG Reading reply sessionid:0x17361e5ba70000d, packet:: clientPath:/marathon/state/group/2/root serverPath:/marathon/state/group/2/root finished:false header:: 12,4 replyHeader:: 12,104,0 request:: '/marathon/state/group/2/root,F response:: #a01218323032302d30372d31385431323a33353a32342e3930315a3222a67365727665721218323032302d30372d31385431323a33353a32342e3930315a,s{24,65,1595075401307,1595075725952,1,2,0,0,64,2,57} (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:36,653] TRACE Trace: GetDataBuilderImpl-Background - 6 ms (org.apache.curator.utils.DefaultTracerDriver:main-EventThread)
2020-07-18 15:23:36,892] DEBUG Reading reply sessionid:0x17361e5ba70000d, packet:: clientPath:/marathon/state/apps/d/server/2020-07-18T12:35:24.901Z serverPath:/marathon/state/apps/d/server/2020-07-18T12:35:24.901Z finished:false header:: 13,4 replyHeader:: 13,104,0 request:: '/marathon/state/apps/d/server/2020-07-18T12:35:24.901Z,F response:: #a72f736572766572122d1201a21707974686f6e202d6d2053696d706c65485454505365727665722024504f5254302a66e6f626f64791812216a4637075731001a99ffffff9affffff99ffffff99ffffff99ffffff99ffffff99ffffffb93f3212a2215a36d656d1001a9900000040403212a2216a46469736b1001a99000000003212a2216a4677075731001a99000000003212a4205a18323032302d30372d31385431323a33353a32342e3930315a68ffffff904e71666666666666fffffff23f7a129000000fffffff03f11000000fffffff03fffffff9010ffffffa81ffffff80ffffffddffffffdb1ffffffb213a12affffffb81ffffffe5ffffff8cffffffb0ffffff8fffffffb62effffffc01ffffffe5ffffff8cffffffb0ffffff8fffffffb62effffffda1e8ffffff904!
e124687474701a3746370ffffff822480100ffffff8821ffffff922281,s{55,64,1595075725569,1595075725915,1,0,0,0,276,0,55} (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:36,895] TRACE Trace: GetDataBuilderImpl-Background - 7 ms (org.apache.curator.utils.DefaultTracerDriver:main-EventThread)
2020-07-18 15:23:38,094] DEBUG Reading reply sessionid:0x17361e5ba70000d, packet:: clientPath:/marathon/state/runtime-configuration/b/id serverPath:/marathon/state/runtime-configuration/b/id finished:false header:: 14,4 replyHeader:: 14,104,0 request:: '/marathon/state/runtime-configuration/b/id,F response:: #7b7d,s{30,99,1595075402795,1595078495546,4,0,0,0,2,0,30} (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:38,095] TRACE Trace: GetDataBuilderImpl-Background - 3 ms (org.apache.curator.utils.DefaultTracerDriver:main-EventThread)
2020-07-18 15:23:38,912] WARN A provider mesosphere.marathon.api.v2.QueueResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider mesosphere.marathon.api.v2.QueueResource will be ignored. (org.glassfish.jersey.internal.inject.Providers:MarathonHttpService STARTING) 2020-07-18 15:23:38,917] WARN A provider mesosphere.marathon.api.v2.LeaderResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider mesosphere.marathon.api.v2.LeaderResource will be ignored. (org.glassfish.jersey.internal.inject.Providers:MarathonHttpService STARTING) 2020-07-18 15:23:38,918] WARN A provider mesosphere.marathon.api.v2.AppsResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider mesosphere.marathon.api.v2.AppsResource will be ignored. (org.glassfish.jersey.internal.inject.Providers:MarathonHttpService STARTING) 2020-07-18 15:23:38,921] WARN A provider mesosphere.marathon.api.v2.GroupsResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider mesosphere.marathon.api.v2.GroupsResource will be ignored. (org.glassfish.jersey.internal.inject.Providers:MarathonHttpService STARTING) 2020-07-18 15:23:38,922] WARN A provider mesosphere.marathon.api.v2.InfoResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider mesosphere.marathon.api.v2.InfoResource will be ignored. (org.glassfish.jersey.internal.inject.Providers:MarathonHttpService STARTING) 2020-07-18 15:23:38,926] WARN A provider mesosphere.marathon.api.v2.PluginsResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider mesosphere.marathon.api.v2.PluginsResource will be ignored. (org.glassfish.jersey.internal.inject.Providers:MarathonHttpService STARTING) 2020-07-18 15:23:38,928] WARN A provider mesosphere.marathon.api.SystemResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider mesosphere.marathon.api.SystemResource will be ignored. (org.glassfish.jersey.internal.inject.Providers:MarathonHttpService STARTING) 2020-07-18 15:23:38,929] WARN A provider mesosphere.marathon.api.v2.TasksResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider mesosphere.marathon.api.v2.TasksResource will be ignored. (org.glassfish.jersey.internal.inject.Providers:MarathonHttpService STARTING) 2020-07-18 15:23:38,930] WARN A provider mesosphere.marathon.api.v2.PodsResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider mesosphere.marathon.api.v2.PodsResource will be ignored. (org.glassfish.jersey.internal.inject.Providers:MarathonHttpService STARTING) 2020-07-18 15:23:38,931] WARN A provider mesosphere.marathon.api.v2.DeploymentsResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider mesosphere.marathon.api.v2.DeploymentsResource will be ignored. (org.glassfish.jersey.internal.inject.Providers:MarathonHttpService STARTING) 2020-07-18 15:23:39,413] DEBUG Reading reply sessionid:0x17361e5ba70000d, packet:: clientPath:/marathon/state/runtime-configuration/b/id serverPath:/marathon/state/runtime-configuration/b/id finished:false header:: 15,5 replyHeader:: 15,105,0 request:: '/marathon/state/runtime-configuration/b/id,#7b7d,-1 response:: s{30,105,1595075402795,1595078619397,5,0,0,0,2,0,30} (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:39,415] TRACE Trace: SetDataBuilderImpl-Background - 30 ms (org.apache.curator.utils.DefaultTracerDriver:main-EventThread)
2020-07-18 15:23:39,447] DEBUG Reading reply sessionid:0x17361e5ba70000d, packet:: clientPath:/marathon/state/internal:storage:version serverPath:/marathon/state/internal:storage:version finished:false header:: 16,4 replyHeader:: 16,105,0 request:: '/marathon/state/internal:storage:version,F response:: #a18696e7465726e616c3a73746f726167653a76657273696f6e122434656462376266342d326166662d343132622d613861342d3964343934366438326336361a981210ffffffc81180201200,s{33,92,1595075402916,1595076012499,1,0,0,0,77,0,33} (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:39,450] TRACE Trace: GetDataBuilderImpl-Background - 5 ms (org.apache.curator.utils.DefaultTracerDriver:main-EventThread)
2020-07-18 15:23:39,457] INFO No migration necessary, already at the target version Version(18, 200, 0, PERSISTENCE_STORE) (mesosphere.marathon.storage.migration.Migration:scala-execution-context-global-41)
2020-07-18 15:23:39,458] INFO Migration successfully applied for version Version(18, 200, 0, PERSISTENCE_STORE) (mesosphere.marathon.storage.migration.Migration:scala-execution-context-global-41)
2020-07-18 15:23:39,463] DEBUG Reading reply sessionid:0x17361e5ba70000d, packet:: clientPath:/marathon/state/group/2/root serverPath:/marathon/state/group/2/root finished:false header:: 17,4 replyHeader:: 17,105,0 request:: '/marathon/state/group/2/root,F response:: #a01218323032302d30372d31385431323a33353a32342e3930315a3222a67365727665721218323032302d30372d31385431323a33353a32342e3930315a,s{24,65,1595075401307,1595075725952,1,2,0,0,64,2,57} (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:39,464] TRACE Trace: GetDataBuilderImpl-Background - 2 ms (org.apache.curator.utils.DefaultTracerDriver:main-EventThread)
2020-07-18 15:23:39,480] INFO Call preDriverStarts callbacks on (mesosphere.marathon.MarathonSchedulerService:pool-1-thread-1)
2020-07-18 15:23:39,482] INFO Finished preDriverStarts callbacks (mesosphere.marathon.MarathonSchedulerService:pool-1-thread-1)
2020-07-18 15:23:39,487] INFO ExpungeOverdueLostTasksActor has started (mesosphere.marathon.core.task.jobs.impl.ExpungeOverdueLostTasksActor:marathon-akka.actor.default-dispatcher-2)
2020-07-18 15:23:39,531] INFO started RateLimiterActor (mesosphere.marathon.core.launchqueue.impl.RateLimiterActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:39,537] DEBUG Reading reply sessionid:0x17361e5ba70000d, packet:: clientPath:/marathon/state/framework-id/b/id serverPath:/marathon/state/framework-id/b/id finished:false header:: 18,4 replyHeader:: 18,105,0 request:: '/marathon/state/framework-id/b/id,F response:: #a2935323632636564392d373065322d346330642d393036342d6162343137333131383430392d30303030,s{39,100,1595075403873,1595078497315,4,0,0,0,43,0,39} (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:39,538] INFO All actors active: [info] * Actor[akka://marathon/user/reviveOffersWhenWanted#131319476]
[info] * Actor[akka://marathon/user/deploymentManager#1520374777]
[info] * Actor[akka://marathon/user/taskKillServiceActor#-1807863507]
[info] * Actor[akka://marathon/user/offersWantedForReconciliation#-736352801]
[info] * Actor[akka://marathon/user/rateLimiter#-1296343297]
[info] * Actor[akka://marathon/user/instanceTracker#-1145393542]
[info] * Actor[akka://marathon/user/killOverdueStagedTasks#-1014414263]
[info] * Actor[akka://marathon/user/offerMatcherLaunchTokens#1705741668]
[info] * Actor[akka://marathon/user/offerMatcherManager#-1248363557]
[info] * Actor[akka://marathon/user/expungeOverdueLostTasks#1659678442]
[info] * Actor[akka://marathon/user/launchQueue#-1257179790] (mesosphere.marathon.core.leadership.impl.LeadershipCoordinatorActor:marathon-akka.actor.default-dispatcher-6)
2020-07-18 15:23:39,539] TRACE Trace: GetDataBuilderImpl-Background - 11 ms (org.apache.curator.utils.DefaultTracerDriver:main-EventThread)
2020-07-18 15:23:39,554] INFO InstanceTrackerActor is starting. Instances loading initiated. (mesosphere.marathon.core.task.tracker.impl.InstanceTrackerActor:marathon-akka.actor.default-dispatcher-2)
2020-07-18 15:23:39,559] INFO no interest in offers for reservation reconciliation anymore. (mesosphere.marathon.core.matcher.reconcile.impl.OffersWantedForReconciliationActor:marathon-akka.actor.default-dispatcher-7)
2020-07-18 15:23:39,597] INFO Started. Will remain interested in offer reconciliation for 17500 milliseconds when needed. (mesosphere.marathon.core.matcher.reconcile.impl.OffersWantedForReconciliationActor:marathon-akka.actor.default-dispatcher-7)
2020-07-18 15:23:39,618] INFO interested in offers for reservation reconciliation because of becoming leader (until 2020-07-18T13:23:57.118Z) (mesosphere.marathon.core.matcher.reconcile.impl.OffersWantedForReconciliationActor:marathon-akka.actor.default-dispatcher-7)
2020-07-18 15:23:39,701] INFO Received offers WANTED notification (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:39,703] INFO => revive offers NOW, canceling any scheduled revives (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:39,722] INFO 2 further revives still needed. Repeating reviveOffers according to --revive_offers_repetitions 3 (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:39,724] INFO => Schedule next revive at 2020-07-18T13:23:44.703Z in 4980 milliseconds, adhering to --min_revive_offers_interval 5000 (ms) (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:39,744] DEBUG Reading reply sessionid:0x17361e5ba70000d, packet:: clientPath:/marathon/state/instance serverPath:/marathon/state/instance finished:false header:: 19,12 replyHeader:: 19,105,0 request:: '/marathon/state/instance,F response:: v{'5},s{79,79,1595075848552,1595075848552,0,1,0,0,0,1,80} (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:39,745] TRACE Trace: GetChildrenBuilderImpl-Background - 12 ms (org.apache.curator.utils.DefaultTracerDriver:main-EventThread)
2020-07-18 15:23:39,769] DEBUG Reading reply sessionid:0x17361e5ba70000d, packet:: clientPath:/marathon/state/instance/5 serverPath:/marathon/state/instance/5 finished:false header:: 20,12 replyHeader:: 20,105,0 request:: '/marathon/state/instance/5,F response:: v{'server.marathon-709bd447-c8f3-11ea-a69d-024206b8be6a},s{80,80,1595075848572,1595075848572,0,1,0,0,0,1,81} (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:39,770] TRACE Trace: GetChildrenBuilderImpl-Background - 7 ms (org.apache.curator.utils.DefaultTracerDriver:main-EventThread)
2020-07-18 15:23:39,846] DEBUG Reading reply sessionid:0x17361e5ba70000d, packet:: clientPath:/marathon/state/instance/5/server.marathon-709bd447-c8f3-11ea-a69d-024206b8be6a serverPath:/marathon/state/instance/5/server.marathon-709bd447-c8f3-11ea-a69d-024206b8be6a finished:false header:: 21,4 replyHeader:: 21,105,0 request:: '/marathon/state/instance/5/server.marathon-709bd447-c8f3-11ea-a69d-024206b8be6a,F response:: #7b22696e7374616e63654964223a7b226964537472696e67223a227365727665722e6d61726174686f6e2d37303962643434372d633866332d313165612d613639642d303234323036623862653661227d2c226167656e74496e666f223a7b22686f7374223a226d30312e6c6f63616c222c226167656e744964223a2235663133393633372d336238342d343631392d386137362d3165383066303366613363652d5330222c22726567696f6e223a22616d733034222c227a6f6e65223a22616d7330342d7261636b32222c2261747472696275746573223a5b5d7d2c227461736b734d6170223a7b227365727665722e37303962643434372d633866332d313165612d613639642d30!
3234323036623862653661223a7b227461736b4964223a227365727665722e37303962643434372d633866332d313165612d613639642d303234323036623862653661222c2272756e5370656356657273696f6e223a22323032302d30372d31385431323a33353a32342e3930315a222c22737461747573223a7b227374616765644174223a22323032302d30372d31385431323a33373a32382e3231305a222c22737461727465644174223a22323032302d30372d31385431323a33373a32392e3531365a222c226d65736f73537461747573223a224369304b4b334e6c636e5a6c636934334d446c695a4451304e79316a4f47597a4c5445785a574574595459355a4330774d6a51794d445a694f474a6c4e6d455141536f70436963315a6a457a4f54597a4e79307a596a67304c5451324d546b744f4745334e6930785a5467775a6a417a5a6d457a59325574557a417870794e5251727645313045364c516f7263325679646d56794c6a63774f574a6b4e4451334c574d345a6a4d744d54466c595331684e6a6c6b4c5441794e4449774e6d4934596d5532595567435768443642656a4141557443757054537264337542774969616b414b45696f51456734784f5449754d5459344c6a45774c6a45314d526a7a776745694a676f6b597a6c6d4d475a6a4d5459744d4751!
784e5330304e6d55344c574a6a4e5749744e44466a4e7a4669596a466a4e7a!
4a68222c22636f6e646974696f6e223a2252756e6e696e67222c226e6574776f726b496e666f223a7b22686f73744e616d65223a226d30312e6c6f63616c222c22686f7374506f727473223a5b33313335345d2c226970416464726573736573223a5b7b22697041646472657373223a223139322e3136382e31302e313531222c2270726f746f636f6c223a2249507634227d5d7d7d7d7d2c2272756e5370656356657273696f6e223a22323032302d30372d31385431323a33353a32342e3930315a222c227374617465223a7b22636f6e646974696f6e223a2252756e6e696e67222c2273696e6365223a22323032302d30372d31385431323a33373a32392e3531365a222c2261637469766553696e6365223a22323032302d30372d31385431323a33373a32392e3531365a222c22676f616c223a2252756e6e696e67227d2c22756e726561636861626c655374726174656779223a7b22696e61637469766541667465725365636f6e6473223a302c22657870756e676541667465725365636f6e6473223a307d7d,s{81,83,1595075848591,1595075849730,2,0,0,0,1186,0,81} (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:39,847] TRACE Trace: GetDataBuilderImpl-Background - 3 ms (org.apache.curator.utils.DefaultTracerDriver:main-EventThread)
2020-07-18 15:23:39,614] INFO Create new Scheduler Driver with frameworkId: Some(value: "5262ced9-70e2-4c0d-9064-ab4173118409-0000"
[info] ) and scheduler mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor@47a7a101 (mesosphere.marathon.MarathonSchedulerDriver$:pool-1-thread-1)
2020-07-18 15:23:40,415] INFO PARTITION_AWARE feature enabled. (mesosphere.marathon.MarathonSchedulerDriver$:pool-1-thread-1)
2020-07-18 15:23:40,417] INFO REGION_AWARE feature enabled (mesosphere.marathon.MarathonSchedulerDriver$:pool-1-thread-1)
2020-07-18 15:23:40,464] DEBUG Start creating new driver (mesosphere.marathon.MarathonSchedulerDriver$:pool-1-thread-1)
2020-07-18 15:23:40,547] DEBUG Finished creating new driver (mesosphere.marathon.MarathonSchedulerDriver$:pool-1-thread-1)
2020-07-18 15:23:40,628] INFO Heartbeat actor activated (mesosphere.marathon.core.heartbeat.HeartbeatActor:marathon-akka.actor.default-dispatcher-7)
[[31merror] I0718 15:23:40.646270 1575166 sched.cpp:239] Version: 1.10.0 [[31merror] I0718 15:23:40.664299 1575176 sched.cpp:343] New master detected at [email protected]:5050 2020-07-18 15:23:40,666] INFO Became active. Accepting event streaming requests. (mesosphere.marathon.core.event.impl.stream.HttpEventStreamActor:marathon-akka.actor.default-dispatcher-4)
2020-07-18 15:23:40,667] INFO Starting scheduler actor (mesosphere.marathon.MarathonSchedulerActor:marathon-akka.actor.default-dispatcher-3)
[[31merror] I0718 15:23:40.674325 1575176 sched.cpp:363] No credentials provided. Attempting to register without authentication [[31merror] I0718 15:23:40.691272 1575172 sched.cpp:751] Framework registered with 5262ced9-70e2-4c0d-9064-ab4173118409-0000
2020-07-18 15:23:40,745] INFO Loaded 1 instances (mesosphere.marathon.core.task.tracker.impl.InstancesLoaderImpl:scala-execution-context-global-45)
2020-07-18 15:23:40,761] DEBUG Reading reply sessionid:0x17361e5ba70000d, packet:: clientPath:/marathon/state/deployment serverPath:/marathon/state/deployment finished:false header:: 22,12 replyHeader:: 22,105,0 request:: '/marathon/state/deployment,F response:: v{'b},s{60,60,1595075725758,1595075725758,0,1,0,0,0,1,61} (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:40,762] TRACE Trace: GetChildrenBuilderImpl-Background - 2 ms (org.apache.curator.utils.DefaultTracerDriver:main-EventThread)
2020-07-18 15:23:40,765] DEBUG Reading reply sessionid:0x17361e5ba70000d, packet:: clientPath:/marathon/state/deployment/b serverPath:/marathon/state/deployment/b finished:false header:: 23,12 replyHeader:: 23,105,0 request:: '/marathon/state/deployment/b,F response:: v{},s{61,61,1595075725774,1595075725774,0,2,0,0,0,0,84} (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:40,768] TRACE Trace: GetChildrenBuilderImpl-Background - 4 ms (org.apache.curator.utils.DefaultTracerDriver:main-EventThread)
2020-07-18 15:23:40,775] INFO Scheduler actor ready (mesosphere.marathon.MarathonSchedulerActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:40,790] INFO Registered heartbeat monitor (mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor:Thread-13)
2020-07-18 15:23:40,791] INFO Registered as 5262ced9-70e2-4c0d-9064-ab4173118409-0000 to master '5262ced9-70e2-4c0d-9064-ab4173118409' (mesosphere.marathon.MarathonScheduler:Thread-13)
2020-07-18 15:23:40,792] INFO Mesos Master version 1.7.2 (mesosphere.marathon.MarathonScheduler:Thread-13)
2020-07-18 15:23:40,803] INFO Instances loading complete. (mesosphere.marathon.core.task.tracker.impl.InstanceTrackerActor:marathon-akka.actor.default-dispatcher-7)
2020-07-18 15:23:40,804] INFO Loaded 1 instances. (mesosphere.marathon.core.task.tracker.impl.InstanceTrackerActor:marathon-akka.actor.default-dispatcher-7)
2020-07-18 15:23:40,831] DEBUG Reading reply sessionid:0x17361e5ba70000d, packet:: clientPath:/marathon/state/framework-id/b/id serverPath:/marathon/state/framework-id/b/id finished:false header:: 24,5 replyHeader:: 24,106,0 request:: '/marathon/state/framework-id/b/id,#a2935323632636564392d373065322d346330642d393036342d6162343137333131383430392d30303030,-1 response:: s{39,106,1595075403873,1595078620815,5,0,0,0,43,0,39} (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:40,832] TRACE Trace: SetDataBuilderImpl-Background - 17 ms (org.apache.curator.utils.DefaultTracerDriver:main-EventThread)
2020-07-18 15:23:40,838] INFO Received reviveOffers notification: SchedulerRegisteredEvent (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:40,838] INFO => Next revive already scheduled at 2020-07-18T13:23:44.703Z not yet due for 3865 milliseconds (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:40,915] DEBUG Adding instances Vector() to the queue (mesosphere.marathon.core.task.termination.impl.KillServiceActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:40,925] INFO Processing offer: offerId 5262ced9-70e2-4c0d-9064-ab4173118409-O12, agentId 5f139637-3b84-4619-8a76-1e80f03fa3ce-S0 (mesosphere.marathon.core.launcher.impl.OfferProcessorImpl:Thread-14)
2020-07-18 15:23:41,080] WARN The following warnings have been detected: WARNING: The (sub)resource method redirectUI in mesosphere.marathon.api.SystemResource contains empty path annotation. [info] (org.glassfish.jersey.internal.Errors:MarathonHttpService STARTING) 2020-07-18 15:23:41,100] INFO Started o.e.j.s.ServletContextHandler@67efd2c2{/,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler:MarathonHttpService STARTING) 2020-07-18 15:23:41,100] DEBUG STARTED @17598ms o.e.j.s.ServletContextHandler@67efd2c2{/,null,AVAILABLE} (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:40,897] INFO reconcile /server with latest version 2020-07-18T12:35:24.901Z for instances: Vector(Instance(instance [server.marathon-709bd447-c8f3-11ea-a69d-024206b8be6a],Some(AgentInfo(m01.local,Some(5f139637-3b84-4619-8a76-1e80f03fa3ce-S0),Some(ams04),Some(ams04-rack2),List())),InstanceState(Running,2020-07-18T12:37:29.516Z,Some(2020-07-18T12:37:29.516Z),None,Running),Map(task [server.709bd447-c8f3-11ea-a69d-024206b8be6a] -> Task(task [server.709bd447-c8f3-11ea-a69d-024206b8be6a],2020-07-18T12:35:24.901Z,Status(2020-07-18T12:37:28.210Z,Some(2020-07-18T12:37:29.516Z),Some(task_id { [info] value: "server.709bd447-c8f3-11ea-a69d-024206b8be6a"
[info] }
[info] state: TASK_RUNNING [info] slave_id { [info] value: "5f139637-3b84-4619-8a76-1e80f03fa3ce-S0"
[info] }
[info] timestamp: 1.595075849267801E9 [info] executor_id { [info] value: "server.709bd447-c8f3-11ea-a69d-024206b8be6a"
[info] }
[info] source: SOURCE_EXECUTOR [info] uuid: "\372\005\350\300\001KB\272\224\322\255\335\356\a\002""
[info] container_status { [info] network_infos {
[info] ip_addresses {
[info] ip_address: "192.168.10.151"
[info] }
[info] }
[info] executor_pid: 24947 [info] container_id {
[info] value: "c9f0fc16-0d15-46e8-bc5b-41c71bb1c72a"
[info] }
[info] }
[info] ),Running,NetworkInfo(m01.local,List(31354),List(protocol: IPv4 [info] ip_address: "192.168.10.151" [info] ))))),AppDefinition(/server,Some(python -m SimpleHTTPServer $PORT0),List(),Some(nobody),Map(),1,Resources(0.1,32.0,0.0,0),,None,Set(),Vector(),Vector(PortDefinition(10000,tcp,Some(http),Map())),false,BackoffStrategy(10000 milliseconds,3600000 milliseconds,1.15),None,Set(),None,Vector(),None,Set(),UpgradeStrategy(1.0,1.0),Map(),Set(),Vector(HostNetwork),FullVersionInfo(2020-07-18T12:35:24.901Z,2020-07-18T12:35:24.901Z,2020-07-18T12:35:24.901Z),Map(),UnreachableEnabled(0 seconds,0 seconds),YoungestFirst,None),None)) (mesosphere.marathon.core.health.impl.MarathonHealthCheckManager:scala-execution-context-global-45)
2020-07-18 15:23:41,154] DEBUG STARTED @17651ms mesosphere.marathon.metrics.current.InstrumentedHandler@665522c2 (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,163] DEBUG starting mesosphere.marathon.api.HttpTransferMetricsHandler@78c1372d (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,163] DEBUG starting mesosphere.marathon.api.HttpTransferMetricsHandler@78c1372d (org.eclipse.jetty.server.handler.AbstractHandler:MarathonHttpService STARTING) 2020-07-18 15:23:41,164] DEBUG STARTED @17661ms mesosphere.marathon.api.HttpTransferMetricsHandler@78c1372d (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,168] DEBUG starting org.eclipse.jetty.server.handler.RequestLogHandler@3c69362a (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,168] DEBUG starting org.eclipse.jetty.server.handler.RequestLogHandler@3c69362a (org.eclipse.jetty.server.handler.AbstractHandler:MarathonHttpService STARTING) 2020-07-18 15:23:41,170] DEBUG starting mesosphere.marathon.api.JettyRequestLog@2412a42b (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,172] DEBUG STARTED @17669ms mesosphere.marathon.api.JettyRequestLog@2412a42b (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,180] DEBUG STARTED @17677ms org.eclipse.jetty.server.handler.RequestLogHandler@3c69362a (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,181] DEBUG STARTED @17678ms org.eclipse.jetty.server.handler.HandlerCollection@2016f509[mesosphere.marathon.metrics.current.InstrumentedHandler@665522c2, mesosphere.marathon.api.HttpTransferMetricsHandler@78c1372d, org.eclipse.jetty.server.handler.RequestLogHandler@3c69362a] (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,183] DEBUG STARTED @17680ms org.eclipse.jetty.server.handler.gzip.GzipHandler@7dd611c8 (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,184] DEBUG starting org.eclipse.jetty.server.handler.ErrorHandler@449e0ef4 (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,184] DEBUG starting org.eclipse.jetty.server.handler.ErrorHandler@449e0ef4 (org.eclipse.jetty.server.handler.AbstractHandler:MarathonHttpService STARTING) 2020-07-18 15:23:41,185] DEBUG STARTED @17682ms org.eclipse.jetty.server.handler.ErrorHandler@449e0ef4 (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,186] DEBUG starting ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,196] INFO addAllFor [/server] version [2020-07-18T12:35:24.901Z] (mesosphere.marathon.core.health.impl.MarathonHealthCheckManager:scala-execution-context-global-45)
2020-07-18 15:23:41,203] DEBUG ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} added {sun.nio.ch.ServerSocketChannelImpl[/0.0.0.0:8080],POJO} (org.eclipse.jetty.util.component.ContainerLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,219] DEBUG starting org.eclipse.jetty.util.thread.ScheduledExecutorScheduler@7569ea63 (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,224] DEBUG STARTED @17721ms org.eclipse.jetty.util.thread.ScheduledExecutorScheduler@7569ea63 (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,225] DEBUG starting HttpConnectionFactory@61c42416[HTTP/1.1] (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,229] DEBUG STARTED @17726ms HttpConnectionFactory@61c42416[HTTP/1.1] (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,230] DEBUG starting SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,240] DEBUG ReservedThreadExecutor@14960f1b{s=0/1,p=0}@SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} (org.eclipse.jetty.util.thread.ReservedThreadExecutor:MarathonHttpService STARTING) 2020-07-18 15:23:41,241] DEBUG SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} added {ReservedThreadExecutor@14960f1b{s=0/1,p=0}@SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080},MANAGED} (org.eclipse.jetty.util.component.ContainerLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,268] DEBUG EatWhatYouKill@2a5f64b7/SelectorProducer@552c4f22/IDLE/ReservedThreadExecutor@14960f1b{s=0/1,p=0}@SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} added {SelectorProducer@552c4f22,POJO} (org.eclipse.jetty.util.component.ContainerLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,277] DEBUG EatWhatYouKill@2a5f64b7/SelectorProducer@552c4f22/IDLE/ReservedThreadExecutor@14960f1b{s=0/1,p=0}@SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} created (org.eclipse.jetty.util.thread.strategy.EatWhatYouKill:MarathonHttpService STARTING) 2020-07-18 15:23:41,282] DEBUG org.eclipse.jetty.io.ManagedSelector@2a6c9d31 id=0 keys=-1 selected=-1 actions=0 added {EatWhatYouKill@2a5f64b7/SelectorProducer@552c4f22/IDLE/ReservedThreadExecutor@14960f1b{s=0/1,p=0}@SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080},MANAGED} (org.eclipse.jetty.util.component.ContainerLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,283] DEBUG SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} added {org.eclipse.jetty.io.ManagedSelector@2a6c9d31 id=0 keys=-1 selected=-1 actions=0,AUTO} (org.eclipse.jetty.util.component.ContainerLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,285] DEBUG starting ReservedThreadExecutor@14960f1b{s=0/1,p=0}@SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,286] DEBUG STARTED @17784ms ReservedThreadExecutor@14960f1b{s=0/1,p=0}@SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,289] DEBUG starting org.eclipse.jetty.io.ManagedSelector@2a6c9d31 id=0 keys=-1 selected=-1 actions=0 (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,293] DEBUG starting EatWhatYouKill@2a5f64b7/SelectorProducer@552c4f22/IDLE/ReservedThreadExecutor@14960f1b{s=0/1,p=0}@SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,294] DEBUG STARTED @17791ms EatWhatYouKill@2a5f64b7/SelectorProducer@552c4f22/IDLE/ReservedThreadExecutor@14960f1b{s=0/1,p=0}@SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,295] DEBUG queue org.eclipse.jetty.io.ManagedSelector$$Lambda$1724/245290883@397cdc2d (org.eclipse.jetty.util.thread.QueuedThreadPool:MarathonHttpService STARTING) 2020-07-18 15:23:41,299] DEBUG run org.eclipse.jetty.io.ManagedSelector$$Lambda$1724/245290883@397cdc2d (org.eclipse.jetty.util.thread.QueuedThreadPool:qtp69670055-29)
2020-07-18 15:23:41,300] DEBUG EatWhatYouKill@2a5f64b7/SelectorProducer@552c4f22/IDLE/ReservedThreadExecutor@14960f1b{s=0/1,p=0}@SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} produce (org.eclipse.jetty.util.thread.strategy.EatWhatYouKill:qtp69670055-29)
2020-07-18 15:23:41,300] DEBUG action=null wakeup=false (org.eclipse.jetty.io.ManagedSelector:qtp69670055-29)
2020-07-18 15:23:41,300] DEBUG Selector sun.nio.ch.EPollSelectorImpl@7b5cf6ca waiting on select (org.eclipse.jetty.io.ManagedSelector:qtp69670055-29)
2020-07-18 15:23:41,303] DEBUG STARTED @17800ms org.eclipse.jetty.io.ManagedSelector@2a6c9d31 id=0 keys=0 selected=0 actions=0 (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,303] DEBUG STARTED @17801ms SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,305] DEBUG ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} added {acceptor-0@2e935b73,POJO} (org.eclipse.jetty.util.component.ContainerLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,307] DEBUG id { [info] value: "5262ced9-70e2-4c0d-9064-ab4173118409-O12"
[info] }
[info] framework_id { [info] value: "5262ced9-70e2-4c0d-9064-ab4173118409-0000"
[info] }
[info] slave_id { [info] value: "5f139637-3b84-4619-8a76-1e80f03fa3ce-S0"
[info] }
[info] hostname: "m01.local" [info] resources { [info] name: "cpus" [info] type: SCALAR [info] scalar {
[info] value: 0.9
[info] }
[info] role: "" [info] } [info] resources { [info] name: "mem" [info] type: SCALAR [info] scalar {
[info] value: 2637.0
[info] }
[info] role: "" [info] } [info] resources { [info] name: "disk" [info] type: SCALAR [info] scalar {
[info] value: 5522.0
[info] }
[info] role: "" [info] } [info] resources { [info] name: "ports" [info] type: RANGES [info] ranges {
[info] range {
[info] begin: 31000
[info] end: 31353
[info] }
[info] range {
[info] begin: 31355
[info] end: 32000
[info] }
[info] }
[info] role: "" [info] } [info] url { [info] scheme: "http" [info] address {
[info] hostname: "m01.local"
[info] ip: "192.168.10.151"
[info] port: 5051
[info] }
[info] path: "/slave(1)" [info] } [info] domain { [info] fault_domain {
[info] region {
[info] name: "ams04"
[info] }
[info] zone {
[info] name: "ams04-rack2"
[info] }
[info] }
[info] }
[info] (mesosphere.marathon.core.launcher.impl.OfferProcessorImpl:Thread-14)
2020-07-18 15:23:41,325] DEBUG queue acceptor-0@2e935b73 (org.eclipse.jetty.util.thread.QueuedThreadPool:MarathonHttpService STARTING) 2020-07-18 15:23:41,328] DEBUG run acceptor-0@2e935b73 (org.eclipse.jetty.util.thread.QueuedThreadPool:qtp69670055-30)
2020-07-18 15:23:41,340] INFO Started ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} (org.eclipse.jetty.server.AbstractConnector:MarathonHttpService STARTING) 2020-07-18 15:23:41,341] DEBUG STARTED @17838ms ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,341] INFO Started @17838ms (org.eclipse.jetty.server.Server:MarathonHttpService STARTING) 2020-07-18 15:23:41,341] DEBUG STARTED @17839ms org.eclipse.jetty.server.Server@140d1230[9.4.8.v20171121] (org.eclipse.jetty.util.component.AbstractLifeCycle:MarathonHttpService STARTING) 2020-07-18 15:23:41,342] INFO All services up and running. (mesosphere.marathon.MarathonApp:main)
2020-07-18 15:23:41,387] INFO No match for:5262ced9-70e2-4c0d-9064-ab4173118409-O12 from:m01.local reason:No offers wanted (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-4)
2020-07-18 15:23:41,408] DEBUG declineOffer(5262ced9-70e2-4c0d-9064-ab4173118409-O12) returned status = DRIVER_RUNNING (mesosphere.marathon.core.launcher.impl.TaskLauncherImpl:scala-execution-context-global-45)
2020-07-18 15:23:41,412] DEBUG Finished processing offer '5262ced9-70e2-4c0d-9064-ab4173118409-O12' (mesosphere.marathon.MarathonScheduler:scala-execution-context-global-45)
2020-07-18 15:23:42,541] DEBUG Current State: LaunchTokens:100 OffersWanted:false Matchers:0 OfferQueues:0 UnprocessedOffers:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-4)
2020-07-18 15:23:42,544] DEBUG Reject Elapsed offers. Unprocessed: 0 Overdue:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-4)
2020-07-18 15:23:44,152] DEBUG Got ping response for sessionid: 0x17361e5ba70000d after 2ms (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:44,731] INFO Received TimedCheck (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:44,733] INFO => revive offers NOW, canceling any scheduled revives (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:44,734] INFO 2 further revives still needed. Repeating reviveOffers according to --revive_offers_repetitions 3 (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:44,734] INFO => Schedule next revive at 2020-07-18T13:23:49.733Z in 4999 milliseconds, adhering to --min_revive_offers_interval 5000 (ms) (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:44,738] INFO Processing offer: offerId 5262ced9-70e2-4c0d-9064-ab4173118409-O13, agentId 5f139637-3b84-4619-8a76-1e80f03fa3ce-S0 (mesosphere.marathon.core.launcher.impl.OfferProcessorImpl:Thread-15)
2020-07-18 15:23:44,742] DEBUG id { [info] value: "5262ced9-70e2-4c0d-9064-ab4173118409-O13"
[info] }
[info] framework_id { [info] value: "5262ced9-70e2-4c0d-9064-ab4173118409-0000"
[info] }
[info] slave_id { [info] value: "5f139637-3b84-4619-8a76-1e80f03fa3ce-S0"
[info] }
[info] hostname: "m01.local" [info] resources { [info] name: "cpus" [info] type: SCALAR [info] scalar {
[info] value: 0.9
[info] }
[info] role: "" [info] } [info] resources { [info] name: "mem" [info] type: SCALAR [info] scalar {
[info] value: 2637.0
[info] }
[info] role: "" [info] } [info] resources { [info] name: "disk" [info] type: SCALAR [info] scalar {
[info] value: 5522.0
[info] }
[info] role: "" [info] } [info] resources { [info] name: "ports" [info] type: RANGES [info] ranges {
[info] range {
[info] begin: 31000
[info] end: 31353
[info] }
[info] range {
[info] begin: 31355
[info] end: 32000
[info] }
[info] }
[info] role: "" [info] } [info] url { [info] scheme: "http" [info] address {
[info] hostname: "m01.local"
[info] ip: "192.168.10.151"
[info] port: 5051
[info] }
[info] path: "/slave(1)" [info] } [info] domain { [info] fault_domain {
[info] region {
[info] name: "ams04"
[info] }
[info] zone {
[info] name: "ams04-rack2"
[info] }
[info] }
[info] }
[info] (mesosphere.marathon.core.launcher.impl.OfferProcessorImpl:Thread-15)
2020-07-18 15:23:44,751] INFO No match for:5262ced9-70e2-4c0d-9064-ab4173118409-O13 from:m01.local reason:No offers wanted (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-2)
2020-07-18 15:23:44,756] DEBUG declineOffer(5262ced9-70e2-4c0d-9064-ab4173118409-O13) returned status = DRIVER_RUNNING (mesosphere.marathon.core.launcher.impl.TaskLauncherImpl:scala-execution-context-global-49)
2020-07-18 15:23:44,757] DEBUG Finished processing offer '5262ced9-70e2-4c0d-9064-ab4173118409-O13' (mesosphere.marathon.MarathonScheduler:scala-execution-context-global-49)
2020-07-18 15:23:45,560] DEBUG Current State: LaunchTokens:100 OffersWanted:false Matchers:0 OfferQueues:0 UnprocessedOffers:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:45,560] DEBUG Reject Elapsed offers. Unprocessed: 0 Overdue:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:47,488] DEBUG Got ping response for sessionid: 0x17361e5ba70000d after 1ms (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:48,540] DEBUG Current State: LaunchTokens:100 OffersWanted:false Matchers:0 OfferQueues:0 UnprocessedOffers:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-2)
2020-07-18 15:23:48,540] DEBUG Reject Elapsed offers. Unprocessed: 0 Overdue:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-2)
2020-07-18 15:23:49,750] INFO Received TimedCheck (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-2)
2020-07-18 15:23:49,750] INFO => revive offers NOW, canceling any scheduled revives (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-2)
2020-07-18 15:23:49,750] INFO 1 further revives still needed. Repeating reviveOffers according to --revive_offers_repetitions 3 (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-2)
2020-07-18 15:23:49,751] INFO => Schedule next revive at 2020-07-18T13:23:54.750Z in 4999 milliseconds, adhering to --min_revive_offers_interval 5000 (ms) (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-2)
2020-07-18 15:23:49,755] INFO Processing offer: offerId 5262ced9-70e2-4c0d-9064-ab4173118409-O14, agentId 5f139637-3b84-4619-8a76-1e80f03fa3ce-S0 (mesosphere.marathon.core.launcher.impl.OfferProcessorImpl:Thread-16)
2020-07-18 15:23:49,766] DEBUG id { [info] value: "5262ced9-70e2-4c0d-9064-ab4173118409-O14"
[info] }
[info] framework_id { [info] value: "5262ced9-70e2-4c0d-9064-ab4173118409-0000"
[info] }
[info] slave_id { [info] value: "5f139637-3b84-4619-8a76-1e80f03fa3ce-S0"
[info] }
[info] hostname: "m01.local" [info] resources { [info] name: "cpus" [info] type: SCALAR [info] scalar {
[info] value: 0.9
[info] }
[info] role: "" [info] } [info] resources { [info] name: "mem" [info] type: SCALAR [info] scalar {
[info] value: 2637.0
[info] }
[info] role: "" [info] } [info] resources { [info] name: "disk" [info] type: SCALAR [info] scalar {
[info] value: 5522.0
[info] }
[info] role: "" [info] } [info] resources { [info] name: "ports" [info] type: RANGES [info] ranges {
[info] range {
[info] begin: 31000
[info] end: 31353
[info] }
[info] range {
[info] begin: 31355
[info] end: 32000
[info] }
[info] }
[info] role: "" [info] } [info] url { [info] scheme: "http" [info] address {
[info] hostname: "m01.local"
[info] ip: "192.168.10.151"
[info] port: 5051
[info] }
[info] path: "/slave(1)" [info] } [info] domain { [info] fault_domain {
[info] region {
[info] name: "ams04"
[info] }
[info] zone {
[info] name: "ams04-rack2"
[info] }
[info] }
[info] }
[info] (mesosphere.marathon.core.launcher.impl.OfferProcessorImpl:Thread-16)
2020-07-18 15:23:50,822] DEBUG Got ping response for sessionid: 0x17361e5ba70000d after 1ms (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:51,540] DEBUG Current State: LaunchTokens:100 OffersWanted:false Matchers:0 OfferQueues:0 UnprocessedOffers:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:51,540] DEBUG Reject Elapsed offers. Unprocessed: 0 Overdue:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:54,158] DEBUG Got ping response for sessionid: 0x17361e5ba70000d after 1ms (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:54,540] DEBUG Current State: LaunchTokens:100 OffersWanted:false Matchers:0 OfferQueues:0 UnprocessedOffers:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-2)
2020-07-18 15:23:54,540] DEBUG Reject Elapsed offers. Unprocessed: 0 Overdue:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-2)
2020-07-18 15:23:54,769] INFO Received TimedCheck (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:54,769] INFO => revive offers NOW, canceling any scheduled revives (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:55,555] DEBUG Receive scale run spec for /server (mesosphere.marathon.MarathonSchedulerActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:55,557] DEBUG Added to lock for run spec: id=/server locks=1 lockedRunSpec=Map(/server -> 1) (mesosphere.marathon.MarathonSchedulerActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:55,571] INFO initiate task reconciliation (mesosphere.marathon.MarathonSchedulerActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:55,572] DEBUG Scale for run spec AppDefinition(/server,Some(python -m SimpleHTTPServer $PORT0),List(),Some(nobody),Map(),1,Resources(0.1,32.0,0.0,0),,None,Set(),Vector(),Vector(PortDefinition(10000,tcp,Some(http),Map())),false,BackoffStrategy(10000 milliseconds,3600000 milliseconds,1.15),None,Set(),None,Vector(),None,Set(),UpgradeStrategy(1.0,1.0),Map(),Set(),Vector(HostNetwork),FullVersionInfo(2020-07-18T12:35:24.901Z,2020-07-18T12:35:24.901Z,2020-07-18T12:35:24.901Z),Map(),UnreachableEnabled(0 seconds,0 seconds),YoungestFirst,None) (mesosphere.marathon.SchedulerActions:scheduler-actions-thread-0)
2020-07-18 15:23:55,616] INFO Already running 1 instances of /server. Not scaling. (mesosphere.marathon.SchedulerActions:scheduler-actions-thread-0)
2020-07-18 15:23:55,632] DEBUG Removed lock for run spec: id=/server locks=0 lockedRunSpec=Map() (mesosphere.marathon.MarathonSchedulerActor:marathon-akka.actor.default-dispatcher-7)
2020-07-18 15:23:55,641] INFO Requesting task reconciliation with the Mesos master (mesosphere.marathon.SchedulerActions:scheduler-actions-thread-0)
2020-07-18 15:23:55,643] DEBUG Tasks to reconcile: Set(task_id { [info] value: "server.709bd447-c8f3-11ea-a69d-024206b8be6a"
[info] }
[info] state: TASK_RUNNING [info] slave_id { [info] value: "5f139637-3b84-4619-8a76-1e80f03fa3ce-S0"
[info] }
[info] timestamp: 1.595075849267801E9 [info] executor_id { [info] value: "server.709bd447-c8f3-11ea-a69d-024206b8be6a"
[info] }
[info] source: SOURCE_EXECUTOR [info] uuid: "\372\005\350\300\001KB\272\224\322\255\335\356\a\002""
[info] container_status { [info] network_infos {
[info] ip_addresses {
[info] ip_address: "192.168.10.151"
[info] }
[info] }
[info] executor_pid: 24947 [info] container_id {
[info] value: "c9f0fc16-0d15-46e8-bc5b-41c71bb1c72a"
[info] }
[info] }
[info] ) (mesosphere.marathon.SchedulerActions:scheduler-actions-thread-0)
2020-07-18 15:23:55,663] INFO task reconciliation has finished (mesosphere.marathon.MarathonSchedulerActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:23:55,668] INFO Received status update for task server.709bd447-c8f3-11ea-a69d-024206b8be6a: TASK_RUNNING (Reconciliation: Latest task state, healthy: false) (mesosphere.marathon.MarathonScheduler:Thread-17)
2020-07-18 15:23:55,676] INFO Message [mesosphere.marathon.MarathonSchedulerActor$TasksReconciled$] from Actor[akka://marathon/user/MarathonScheduler/$a#-340391633] to Actor[akka://marathon/deadLetters] was not delivered. [1] dead letters encountered. If this is not an expected behavior, then [Actor[akka://marathon/deadLetters]] may have terminated unexpectedly, This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. (akka.actor.DeadLetterActorRef:marathon-akka.actor.default-dispatcher-2)
2020-07-18 15:23:55,678] INFO Received status update for task server.709bd447-c8f3-11ea-a69d-024206b8be6a: TASK_RUNNING (Reconciliation: Latest task state, healthy: false) (mesosphere.marathon.MarathonScheduler:Thread-18)
2020-07-18 15:23:57,130] INFO no interest in offers for reservation reconciliation anymore. (mesosphere.marathon.core.matcher.reconcile.impl.OffersWantedForReconciliationActor:marathon-akka.actor.default-dispatcher-2)
2020-07-18 15:23:57,133] INFO Received offers NOT WANTED notification, canceling 0 revives (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-5)
2020-07-18 15:23:57,135] INFO => Suppress offers NOW (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-5)
2020-07-18 15:23:57,491] DEBUG Got ping response for sessionid: 0x17361e5ba70000d after 1ms (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:23:57,540] DEBUG Current State: LaunchTokens:100 OffersWanted:false Matchers:0 OfferQueues:0 UnprocessedOffers:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-5)
2020-07-18 15:23:57,540] DEBUG Reject Elapsed offers. Unprocessed: 0 Overdue:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-5)
2020-07-18 15:24:00,540] DEBUG Current State: LaunchTokens:100 OffersWanted:false Matchers:0 OfferQueues:0 UnprocessedOffers:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-5)
2020-07-18 15:24:00,540] DEBUG Reject Elapsed offers. Unprocessed: 0 Overdue:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-5)
2020-07-18 15:24:00,828] DEBUG Got ping response for sessionid: 0x17361e5ba70000d after 1ms (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:24:03,540] DEBUG Current State: LaunchTokens:100 OffersWanted:false Matchers:0 OfferQueues:0 UnprocessedOffers:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-5)
2020-07-18 15:24:03,540] DEBUG Reject Elapsed offers. Unprocessed: 0 Overdue:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-5)
2020-07-18 15:24:04,164] DEBUG Got ping response for sessionid: 0x17361e5ba70000d after 1ms (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:24:06,540] DEBUG Current State: LaunchTokens:100 OffersWanted:false Matchers:0 OfferQueues:0 UnprocessedOffers:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-2)
2020-07-18 15:24:06,540] DEBUG Reject Elapsed offers. Unprocessed: 0 Overdue:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-2)
2020-07-18 15:24:07,499] DEBUG Got ping response for sessionid: 0x17361e5ba70000d after 1ms (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:24:09,540] DEBUG Current State: LaunchTokens:100 OffersWanted:false Matchers:0 OfferQueues:0 UnprocessedOffers:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-2)
2020-07-18 15:24:09,540] DEBUG Reject Elapsed offers. Unprocessed: 0 Overdue:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-2)
2020-07-18 15:24:09,630] DEBUG Checking for overdue tasks (mesosphere.marathon.core.task.jobs.impl.OverdueInstancesActor$Support:marathon-akka.actor.default-dispatcher-2)
2020-07-18 15:24:10,700] INFO Prompting Mesos for a heartbeat via explicit task reconciliation (mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor$$anon$1:marathon-akka.actor.default-dispatcher-2)
2020-07-18 15:24:10,703] INFO Received fake heartbeat task-status update (mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor:Thread-19)
2020-07-18 15:24:10,837] DEBUG Got ping response for sessionid: 0x17361e5ba70000d after 3ms (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:24:12,540] DEBUG Current State: LaunchTokens:100 OffersWanted:false Matchers:0 OfferQueues:0 UnprocessedOffers:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-5)
2020-07-18 15:24:12,540] DEBUG Reject Elapsed offers. Unprocessed: 0 Overdue:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-5)
2020-07-18 15:24:14,172] DEBUG Got ping response for sessionid: 0x17361e5ba70000d after 1ms (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:24:14,620] DEBUG Checking for overdue tasks (mesosphere.marathon.core.task.jobs.impl.OverdueInstancesActor$Support:marathon-akka.actor.default-dispatcher-2)
2020-07-18 15:24:15,540] DEBUG Current State: LaunchTokens:100 OffersWanted:false Matchers:0 OfferQueues:0 UnprocessedOffers:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-2)
2020-07-18 15:24:15,540] DEBUG Reject Elapsed offers. Unprocessed: 0 Overdue:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-2)
2020-07-18 15:24:17,510] DEBUG Got ping response for sessionid: 0x17361e5ba70000d after 1ms (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:24:18,540] DEBUG Current State: LaunchTokens:100 OffersWanted:false Matchers:0 OfferQueues:0 UnprocessedOffers:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:24:18,540] DEBUG Reject Elapsed offers. Unprocessed: 0 Overdue:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:24:19,620] DEBUG Checking for overdue tasks (mesosphere.marathon.core.task.jobs.impl.OverdueInstancesActor$Support:marathon-akka.actor.default-dispatcher-3)
2020-07-18 15:24:20,847] DEBUG Got ping response for sessionid: 0x17361e5ba70000d after 1ms (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:24:21,540] DEBUG Current State: LaunchTokens:100 OffersWanted:false Matchers:0 OfferQueues:0 UnprocessedOffers:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-5)
2020-07-18 15:24:21,540] DEBUG Reject Elapsed offers. Unprocessed: 0 Overdue:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-5)
2020-07-18 15:24:24,181] DEBUG Got ping response for sessionid: 0x17361e5ba70000d after 1ms (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
2020-07-18 15:24:24,550] DEBUG Current State: LaunchTokens:100 OffersWanted:false Matchers:0 OfferQueues:0 UnprocessedOffers:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-7)
2020-07-18 15:24:24,550] DEBUG Reject Elapsed offers. Unprocessed: 0 Overdue:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-7)
2020-07-18 15:24:24,620] DEBUG Checking for overdue tasks (mesosphere.marathon.core.task.jobs.impl.OverdueInstancesActor$Support:marathon-akka.actor.default-dispatcher-5)
2020-07-18 15:24:25,720] INFO Prompting Mesos for a heartbeat via explicit task reconciliation (mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor$$anon$1:marathon-akka.actor.default-dispatcher-7)
2020-07-18 15:24:25,724] INFO Received fake heartbeat task-status update (mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor:Thread-20)
2020-07-18 15:24:27,313] DEBUG Queued change org.eclipse.jetty.io.ManagedSelector$Accept@33bbbb5d on org.eclipse.jetty.io.ManagedSelector@2a6c9d31 id=0 keys=0 selected=0 actions=0 (org.eclipse.jetty.io.ManagedSelector:qtp69670055-30-acceptor-0@2e935b73-ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080})
2020-07-18 15:24:27,314] DEBUG Selector sun.nio.ch.EPollSelectorImpl@7b5cf6ca woken up from select, 0/0 selected (org.eclipse.jetty.io.ManagedSelector:qtp69670055-29)
2020-07-18 15:24:27,314] DEBUG Selector sun.nio.ch.EPollSelectorImpl@7b5cf6ca processing 0 keys, 1 actions (org.eclipse.jetty.io.ManagedSelector:qtp69670055-29)
2020-07-18 15:24:27,314] DEBUG action=org.eclipse.jetty.io.ManagedSelector$Accept@33bbbb5d wakeup=false (org.eclipse.jetty.io.ManagedSelector:qtp69670055-29)
2020-07-18 15:24:27,315] DEBUG EatWhatYouKill@2a5f64b7/SelectorProducer@552c4f22/PRODUCING/ReservedThreadExecutor@14960f1b{s=0/1,p=0}@SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} t=org.eclipse.jetty.io.ManagedSelector$Accept@33bbbb5d/NON_BLOCKING (org.eclipse.jetty.util.thread.strategy.EatWhatYouKill:qtp69670055-29)
2020-07-18 15:24:27,315] DEBUG EatWhatYouKill@2a5f64b7/SelectorProducer@552c4f22/PRODUCING/ReservedThreadExecutor@14960f1b{s=0/1,p=0}@SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} PC t=org.eclipse.jetty.io.ManagedSelector$Accept@33bbbb5d (org.eclipse.jetty.util.thread.strategy.EatWhatYouKill:qtp69670055-29)
2020-07-18 15:24:27,316] DEBUG Queued change org.eclipse.jetty.io.ManagedSelector$CreateEndPoint@135cb56b on org.eclipse.jetty.io.ManagedSelector@2a6c9d31 id=0 keys=1 selected=0 actions=0 (org.eclipse.jetty.io.ManagedSelector:qtp69670055-29)
2020-07-18 15:24:27,316] DEBUG Forcing selection, actions=1 (org.eclipse.jetty.io.ManagedSelector:qtp69670055-29)
2020-07-18 15:24:27,317] DEBUG action=null wakeup=true (org.eclipse.jetty.io.ManagedSelector:qtp69670055-29)
2020-07-18 15:24:27,317] DEBUG Selector sun.nio.ch.EPollSelectorImpl@7b5cf6ca waiting on select (org.eclipse.jetty.io.ManagedSelector:qtp69670055-29)
2020-07-18 15:24:27,317] DEBUG Selector sun.nio.ch.EPollSelectorImpl@7b5cf6ca woken up from select, 0/1 selected (org.eclipse.jetty.io.ManagedSelector:qtp69670055-29)
2020-07-18 15:24:27,317] DEBUG Selector sun.nio.ch.EPollSelectorImpl@7b5cf6ca processing 0 keys, 1 actions (org.eclipse.jetty.io.ManagedSelector:qtp69670055-29)
2020-07-18 15:24:27,317] DEBUG action=org.eclipse.jetty.io.ManagedSelector$CreateEndPoint@135cb56b wakeup=false (org.eclipse.jetty.io.ManagedSelector:qtp69670055-29)
2020-07-18 15:24:27,318] DEBUG EatWhatYouKill@2a5f64b7/SelectorProducer@552c4f22/PRODUCING/ReservedThreadExecutor@14960f1b{s=0/1,p=0}@SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} t=org.eclipse.jetty.io.ManagedSelector$CreateEndPoint@135cb56b/BLOCKING (org.eclipse.jetty.util.thread.strategy.EatWhatYouKill:qtp69670055-29)
2020-07-18 15:24:27,323] DEBUG ReservedThreadExecutor@14960f1b{s=0/1,p=0}@SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} tryExecute EatWhatYouKill@2a5f64b7/SelectorProducer@552c4f22/PRODUCING/ReservedThreadExecutor@14960f1b{s=0/1,p=0}@SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} (org.eclipse.jetty.util.thread.ReservedThreadExecutor:qtp69670055-29)
2020-07-18 15:24:27,327] DEBUG ReservedThreadExecutor@14960f1b{s=0/1,p=1}@SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} startReservedThread p=1 (org.eclipse.jetty.util.thread.ReservedThreadExecutor:qtp69670055-29)
2020-07-18 15:24:27,329] DEBUG queue ReservedThreadExecutor@14960f1b{s=0/1,p=1}@SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}@7e59db56 (org.eclipse.jetty.util.thread.QueuedThreadPool:qtp69670055-29)
2020-07-18 15:24:27,338] DEBUG run ReservedThreadExecutor@14960f1b{s=0/1,p=1}@SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}@7e59db56 (org.eclipse.jetty.util.thread.QueuedThreadPool:qtp69670055-31)
2020-07-18 15:24:27,338] DEBUG ReservedThreadExecutor@14960f1b{s=1/1,p=1}@SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}@7e59db56 started (org.eclipse.jetty.util.thread.ReservedThreadExecutor:qtp69670055-31)
2020-07-18 15:24:27,339] DEBUG ReservedThreadExecutor@14960f1b{s=1/1,p=0}@SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}@7e59db56 waiting (org.eclipse.jetty.util.thread.ReservedThreadExecutor:qtp69670055-31)
2020-07-18 15:24:27,339] DEBUG EatWhatYouKill@2a5f64b7/SelectorProducer@552c4f22/PRODUCING/ReservedThreadExecutor@14960f1b{s=1/1,p=0}@SelectorManager@ServerConnector@3a175162{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} PEC t=org.eclipse.jetty.io.ManagedSelector$CreateEndPoint@135cb56b (org.eclipse.jetty.util.thread.strategy.EatWhatYouKill:qtp69670055-29)
2020-07-18 15:24:27,341] DEBUG queue org.eclipse.jetty.io.ManagedSelector$CreateEndPoint@135cb56b (org.eclipse.jetty.util.thread.QueuedThreadPool:qtp69670055-29)
2020-07-18 15:24:27,342] DEBUG run org.eclipse.jetty.io.ManagedSelector$CreateEndPoint@135cb56b (org.eclipse.jetty.util.thread.QueuedThreadPool:qtp69670055-32)
2020-07-18 15:24:27,346] DEBUG Forcing selection, actions=0 (org.eclipse.jetty.io.ManagedSelector:qtp69670055-29)
2020-07-18 15:24:27,346] DEBUG action=null wakeup=false (org.eclipse.jetty.io.ManagedSelector:qtp69670055-29)
2020-07-18 15:24:27,346] DEBUG Selector sun.nio.ch.EPollSelectorImpl@7b5cf6ca waiting on select (org.eclipse.jetty.io.ManagedSelector:qtp69670055-29)
2020-07-18 15:24:27,368] DEBUG SocketChannelEndPoint@bd72e54{/192.168.10.219:53337<->/192.168.10.22:8080,OPEN,fill=-,flush=-,to=16/30000}{io=0/0,kio=0,kro=0}->
[[33mwarn] Canceling execution... [[31merror] I0718 15:24:56.504509 1575186 sched.cpp:2166] Asked to stop the driver [[31merror] I0718 15:24:56.504631 1575172 sched.cpp:1204] Stopping framework 5262ced9-70e2-4c0d-9064-ab4173118409-0000 2020-07-18 15:24:56,472] INFO Shutting down services (mesosphere.marathon.MarathonApp:shutdownHook3) [[31merror] Total time: 118 s, completed Jul 18, 2020 3:24:58 PM
This is the v2/info request compared between 1.7.236 an 1.10.25
1.7.236
[info] [2020-07-18 23:00:22,571] DEBUG REQUEST for //test2.local:7071/v2/info on HttpChannelOverHttp@1eecfa5c{r=1,c=false,a=IDLE,uri=//test2.local:7071/v2/info}
[info] GET //test2.local:7071/v2/info HTTP/1.1
[info] Host: test2.local:7071
[info] User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0
[info] Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
[info] Accept-Language: en-US,en;q=0.5
[info] Accept-Encoding: gzip, deflate
[info] DNT: 1
[info] Connection: keep-alive
[info] Upgrade-Insecure-Requests: 1
[info] Cache-Control: max-age=0
[info] (org.eclipse.jetty.server.HttpChannel:qtp1640832113-32)
[info] [2020-07-18 23:00:22,571] DEBUG HttpChannelOverHttp@1eecfa5c{r=1,c=false,a=IDLE,uri=//test2.local:7071/v2/info} onContentComplete (org.eclipse.jetty.server.HttpChannel:qtp1640832113-32)
[info] [2020-07-18 23:00:22,571] DEBUG HttpChannelOverHttp@1eecfa5c{r=1,c=false,a=IDLE,uri=//test2.local:7071/v2/info} onRequestComplete (org.eclipse.jetty.server.HttpChannel:qtp1640832113-32)
[info] [2020-07-18 23:00:22,572] DEBUG HttpInputOverHTTP@7a398745[c=0,q=0,[0]=null,s=STREAM] addContent EOF (org.eclipse.jetty.server.HttpInput:qtp1640832113-32)
[info] [2020-07-18 23:00:22,573] DEBUG HttpConnection@df1bc7a[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7caf188e{s=START}]=>HttpChannelOverHttp@1eecfa5c{r=1,c=false,a=IDLE,uri=//test2.local:7071/v2/info}<-SocketChannelEndPoint@3d798ac0{/192.168.10.219:57151<->/192.168.10.22:7071,OPEN,fill=-,flush=-,to=23/30000}{io=0/0,kio=0,kro=1}->HttpConnection@df1bc7a[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7caf188e{s=START}]=>HttpChannelOverHttp@1eecfa5c{r=1,c=false,a=IDLE,uri=//test2.local:7071/v2/info} parsed true HttpParser{s=END,0 of -1} (org.eclipse.jetty.server.HttpConnection:qtp1640832113-32)
[info] [2020-07-18 23:00:22,573] DEBUG releaseRequestBuffer HttpConnection@df1bc7a[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7caf188e{s=START}]=>HttpChannelOverHttp@1eecfa5c{r=1,c=false,a=IDLE,uri=//test2.local:7071/v2/info}<-SocketChannelEndPoint@3d798ac0{/192.168.10.219:57151<->/192.168.10.22:7071,OPEN,fill=-,flush=-,to=23/30000}{io=0/0,kio=0,kro=1}->HttpConnection@df1bc7a[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7caf188e{s=START}]=>HttpChannelOverHttp@1eecfa5c{r=1,c=false,a=IDLE,uri=//test2.local:7071/v2/info} (org.eclipse.jetty.server.HttpConnection:qtp1640832113-32)
[info] [2020-07-18 23:00:22,573] DEBUG HttpChannelOverHttp@1eecfa5c{r=1,c=false,a=IDLE,uri=//test2.local:7071/v2/info} handle //test2.local:7071/v2/info (org.eclipse.jetty.server.HttpChannel:qtp1640832113-32)
[info] [2020-07-18 23:00:22,574] DEBUG handling HttpChannelState@6c10396e{s=IDLE a=NOT_ASYNC i=true r=IDLE w=false} (org.eclipse.jetty.server.HttpChannelState:qtp1640832113-32)
[info] [2020-07-18 23:00:22,574] DEBUG HttpChannelOverHttp@1eecfa5c{r=1,c=false,a=DISPATCHED,uri=//test2.local:7071/v2/info} action DISPATCH (org.eclipse.jetty.server.HttpChannel:qtp1640832113-32)
[info] [2020-07-18 23:00:22,574] DEBUG REQUEST GET /v2/info on HttpChannelOverHttp@1eecfa5c{r=1,c=false,a=DISPATCHED,uri=//test2.local:7071/v2/info} (org.eclipse.jetty.server.Server:qtp1640832113-32)
[info] [2020-07-18 23:00:22,574] DEBUG org.eclipse.jetty.server.handler.gzip.GzipHandler@3c87e6b7 handle Request(GET //test2.local:7071/v2/info)@5f20bc38 in null (org.eclipse.jetty.server.handler.gzip.GzipHandler:qtp1640832113-32)
[info] [2020-07-18 23:00:22,574] DEBUG scope null||/v2/info @ o.e.j.s.ServletContextHandler@b52b755{/,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler:qtp1640832113-32)
[info] [2020-07-18 23:00:22,574] DEBUG context=||/v2/info @ o.e.j.s.ServletContextHandler@b52b755{/,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler:qtp1640832113-32)
[info] [2020-07-18 23:00:22,574] DEBUG servlet ||/v2/info -> org.glassfish.jersey.servlet.ServletContainer-5e8507f1@9bff24ad==org.glassfish.jersey.servlet.ServletContainer,jsp=null,order=-1,inst=true (org.eclipse.jetty.servlet.ServletHandler:qtp1640832113-32)
[info] [2020-07-18 23:00:22,574] DEBUG chain=mesosphere.marathon.api.LeaderProxyFilter-41184371->mesosphere.marathon.api.LimitConcurrentRequestsFilter-608b1fd2->mesosphere.marathon.api.CORSFilter-56382bc9->mesosphere.marathon.api.CacheDisablingFilter-591a4f8e->org.glassfish.jersey.servlet.ServletContainer-5e8507f1@9bff24ad==org.glassfish.jersey.servlet.ServletContainer,jsp=null,order=-1,inst=true (org.eclipse.jetty.servlet.ServletHandler:qtp1640832113-32)
[info] [2020-07-18 23:00:22,575] DEBUG call filter mesosphere.marathon.api.LeaderProxyFilter-41184371 (org.eclipse.jetty.servlet.ServletHandler:qtp1640832113-32)
[info] [2020-07-18 23:00:22,575] DEBUG call filter mesosphere.marathon.api.LimitConcurrentRequestsFilter-608b1fd2 (org.eclipse.jetty.servlet.ServletHandler:qtp1640832113-32)
[info] [2020-07-18 23:00:22,575] DEBUG call filter mesosphere.marathon.api.CORSFilter-56382bc9 (org.eclipse.jetty.servlet.ServletHandler:qtp1640832113-32)
[info] [2020-07-18 23:00:22,575] DEBUG call filter mesosphere.marathon.api.CacheDisablingFilter-591a4f8e (org.eclipse.jetty.servlet.ServletHandler:qtp1640832113-32)
[info] [2020-07-18 23:00:22,575] DEBUG call servlet org.glassfish.jersey.servlet.ServletContainer-5e8507f1@9bff24ad==org.glassfish.jersey.servlet.ServletContainer,jsp=null,order=-1,inst=true (org.eclipse.jetty.servlet.ServletHandler:qtp1640832113-32)
[info] [2020-07-18 23:00:22,585] DEBUG startAsync HttpChannelState@6c10396e{s=DISPATCHED a=NOT_ASYNC i=true r=IDLE w=false} (org.eclipse.jetty.server.HttpChannelState:qtp1640832113-32)
[info] [2020-07-18 23:00:22,594] DEBUG handled=true async=true committed=false on HttpChannelOverHttp@1eecfa5c{r=1,c=false,a=DISPATCHED,uri=//test2.local:7071/v2/info} (org.eclipse.jetty.server.Server:qtp1640832113-32)
[info] [2020-07-18 23:00:22,595] DEBUG unhandle HttpChannelState@6c10396e{s=DISPATCHED a=STARTED i=true r=IDLE w=false} (org.eclipse.jetty.server.HttpChannelState:qtp1640832113-32)
[info] [2020-07-18 23:00:22,596] DEBUG HttpChannelOverHttp@1eecfa5c{r=1,c=false,a=ASYNC_WAIT,uri=//test2.local:7071/v2/info} action WAIT (org.eclipse.jetty.server.HttpChannel:qtp1640832113-32)
[info] [2020-07-18 23:00:22,597] DEBUG HttpChannelOverHttp@1eecfa5c{r=1,c=false,a=ASYNC_WAIT,uri=//test2.local:7071/v2/info} handle exit, result WAIT (org.eclipse.jetty.server.HttpChannel:qtp1640832113-32)
[info] [2020-07-18 23:00:22,598] DEBUG HttpConnection@df1bc7a[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7caf188e{s=START}]=>HttpChannelOverHttp@1eecfa5c{r=1,c=false,a=ASYNC_WAIT,uri=//test2.local:7071/v2/info}<-SocketChannelEndPoint@3d798ac0{/192.168.10.219:57151<->/192.168.10.22:7071,OPEN,fill=-,flush=-,to=48/30000}{io=0/0,kio=0,kro=1}->HttpConnection@df1bc7a[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7caf188e{s=START}]=>HttpChannelOverHttp@1eecfa5c{r=1,c=false,a=ASYNC_WAIT,uri=//test2.local:7071/v2/info} onFillable exit HttpChannelState@6c10396e{s=ASYNC_WAIT a=STARTED i=false r=IDLE w=false} null (org.eclipse.jetty.server.HttpConnection:qtp1640832113-32)
[info] [2020-07-18 23:00:22,598] DEBUG ReservedThreadExecutor@298e6c95{s=1/1,p=0}@SelectorManager@ServerConnector@75ae4a1f{HTTP/1.1,[http/1.1]}{0.0.0.0:7071}@40fe012 size 1 > capacity (org.eclipse.jetty.util.thread.ReservedThreadExecutor:qtp1640832113-32)
[info] [2020-07-18 23:00:22,598] DEBUG ran ReservedThreadExecutor@298e6c95{s=1/1,p=0}@SelectorManager@ServerConnector@75ae4a1f{HTTP/1.1,[http/1.1]}{0.0.0.0:7071}@40fe012 (org.eclipse.jetty.util.thread.QueuedThreadPool:qtp1640832113-32)
[info] [2020-07-18 23:00:22,600] DEBUG org.eclipse.jetty.server.handler.gzip.GzipHttpOutputInterceptor@262a7e31 compressing java.util.zip.Deflater@5f15d197 (org.eclipse.jetty.server.handler.gzip.GzipHttpOutputInterceptor:scala-execution-context-global-235)
[info] [2020-07-18 23:00:22,600] DEBUG sendResponse info=null content=HeapByteBuffer@2c81139b[p=0,l=755,c=32768,r=755]={<<<\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x00}UM\x8f\x9b0\x10...|\xBbM\xDf\xDf\xFf\x03\x07\xD7aL\x96\x06\x00\x00>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} complete=true committing=true callback=GzipBufferCB@7e0724e3 (org.eclipse.jetty.server.HttpChannel:scala-execution-context-global-235)
[info] [2020-07-18 23:00:22,600] DEBUG COMMIT for /v2/info on HttpChannelOverHttp@1eecfa5c{r=1,c=true,a=ASYNC_WAIT,uri=//test2.local:7071/v2/info}
[info] 200 OK HTTP/1.1
[info] Date: Sat, 18 Jul 2020 21:00:22 GMT
[info] X-Marathon-Leader: http://test2.local:7071
[info] X-Frame-Options: DENY
[info] X-XSS-Protection: 1; mode=block
[info] Cache-Control: no-cache, no-store, must-revalidate
[info] Pragma: no-cache
[info] Expires: 0
[info] Content-Type: application/json
[info] Vary: Accept-Encoding, User-Agent
[info] Content-Encoding: gzip
[info] (org.eclipse.jetty.server.HttpChannel:scala-execution-context-global-235)
[info] [2020-07-18 23:00:22,600] DEBUG org.eclipse.jetty.server.HttpConnection$SendCallback@721c56b[PROCESSING][i=HTTP/1.1{s=200,h=10,cl=-1},cb=org.eclipse.jetty.server.HttpChannel$CommitCallback@480011e0] generate: NEED_HEADER (null,[p=0,l=755,c=32768,r=755],true)@START (org.eclipse.jetty.server.HttpConnection:scala-execution-context-global-235)
[info] [2020-07-18 23:00:22,600] DEBUG generateHeaders HTTP/1.1{s=200,h=10,cl=-1} last=true content=HeapByteBuffer@2c81139b[p=0,l=755,c=32768,r=755]={<<<\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x00}UM\x8f\x9b0\x10...|\xBbM\xDf\xDf\xFf\x03\x07\xD7aL\x96\x06\x00\x00>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} (org.eclipse.jetty.http.HttpGenerator:scala-execution-context-global-235)
[info] [2020-07-18 23:00:22,600] DEBUG Date: Sat, 18 Jul 2020 21:00:22 GMT
[info] X-Marathon-Leader: http://test2.local:7071
[info] X-Frame-Options: DENY
[info] X-XSS-Protection: 1; mode=block
[info] Cache-Control: no-cache, no-store, must-revalidate
[info] Pragma: no-cache
[info
1.10.25
[2020-07-18 21:40:15,433] DEBUG REQUEST for //test2.local:7070/v2/info on HttpChannelOverHttp@5cb30625{r=1,c=false,a=IDLE,uri=//test2.local:7070/v2/info}
GET //test2.local:7070/v2/info HTTP/1.1
Host: test2.local:7070
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Connection: keep-alive
Upgrade-Insecure-Requests: 1
(org.eclipse.jetty.server.HttpChannel:qtp2094310615-27)
[2020-07-18 21:40:15,436] DEBUG HttpChannelOverHttp@5cb30625{r=1,c=false,a=IDLE,uri=//test2.local:7070/v2/info} onContentComplete (org.eclipse.jetty.server.HttpChannel:qtp2094310615-27)
[2020-07-18 21:40:15,438] DEBUG HttpChannelOverHttp@5cb30625{r=1,c=false,a=IDLE,uri=//test2.local:7070/v2/info} onRequestComplete (org.eclipse.jetty.server.HttpChannel:qtp2094310615-27)
[2020-07-18 21:40:15,439] DEBUG HttpInputOverHTTP@653101ee[c=0,q=0,[0]=null,s=STREAM] addContent EOF (org.eclipse.jetty.server.HttpInput:qtp2094310615-27)
[2020-07-18 21:40:15,440] DEBUG HttpConnection@475cea3e[p=HttpParser{s=END,0 of -1},g=HttpGenerator@4d2dab43{s=START}]=>HttpChannelOverHttp@5cb30625{r=1,c=false,a=IDLE,uri=//test2.local:7070/v2/info}<-SocketChannelEndPoint@1afde438{/192.168.10.219:57323<->/192.168.10.22:7070,OPEN,fill=-,flush=-,to=44/30000}{io=0/0,kio=0,kro=1}->HttpConnection@475cea3e[p=HttpParser{s=END,0 of -1},g=HttpGenerator@4d2dab43{s=START}]=>HttpChannelOverHttp@5cb30625{r=1,c=false,a=IDLE,uri=//test2.local:7070/v2/info} parsed true HttpParser{s=END,0 of -1} (org.eclipse.jetty.server.HttpConnection:qtp2094310615-27)
[2020-07-18 21:40:15,441] DEBUG releaseRequestBuffer HttpConnection@475cea3e[p=HttpParser{s=END,0 of -1},g=HttpGenerator@4d2dab43{s=START}]=>HttpChannelOverHttp@5cb30625{r=1,c=false,a=IDLE,uri=//test2.local:7070/v2/info}<-SocketChannelEndPoint@1afde438{/192.168.10.219:57323<->/192.168.10.22:7070,OPEN,fill=-,flush=-,to=45/30000}{io=0/0,kio=0,kro=1}->HttpConnection@475cea3e[p=HttpParser{s=END,0 of -1},g=HttpGenerator@4d2dab43{s=START}]=>HttpChannelOverHttp@5cb30625{r=1,c=false,a=IDLE,uri=//test2.local:7070/v2/info} (org.eclipse.jetty.server.HttpConnection:qtp2094310615-27)
[2020-07-18 21:40:15,442] DEBUG HttpChannelOverHttp@5cb30625{r=1,c=false,a=IDLE,uri=//test2.local:7070/v2/info} handle //test2.local:7070/v2/info (org.eclipse.jetty.server.HttpChannel:qtp2094310615-27)
[2020-07-18 21:40:15,443] DEBUG handling HttpChannelState@29476a6{s=IDLE a=NOT_ASYNC i=true r=IDLE w=false} (org.eclipse.jetty.server.HttpChannelState:qtp2094310615-27)
[2020-07-18 21:40:15,445] DEBUG HttpChannelOverHttp@5cb30625{r=1,c=false,a=DISPATCHED,uri=//test2.local:7070/v2/info} action DISPATCH (org.eclipse.jetty.server.HttpChannel:qtp2094310615-27)
[2020-07-18 21:40:15,454] DEBUG REQUEST GET /v2/info on HttpChannelOverHttp@5cb30625{r=1,c=false,a=DISPATCHED,uri=//test2.local:7070/v2/info} (org.eclipse.jetty.server.Server:qtp2094310615-27)
[2020-07-18 21:40:15,455] DEBUG org.eclipse.jetty.server.handler.gzip.GzipHandler@2d9dff65 handle Request(GET //test2.local:7070/v2/info)@5ea971e5 in null (org.eclipse.jetty.server.handler.gzip.GzipHandler:qtp2094310615-27)
[2020-07-18 21:40:15,457] DEBUG scope null||/v2/info @ o.e.j.s.ServletContextHandler@6920614{/,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler:qtp2094310615-27)
[2020-07-18 21:40:15,459] DEBUG context=||/v2/info @ o.e.j.s.ServletContextHandler@6920614{/,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler:qtp2094310615-27)
[2020-07-18 21:40:15,460] DEBUG servlet ||/v2/info -> org.glassfish.jersey.servlet.ServletContainer-5d1e0fbb@5dcc1567==org.glassfish.jersey.servlet.ServletContainer,jsp=null,order=-1,inst=true (org.eclipse.jetty.servlet.ServletHandler:qtp2094310615-27)
[2020-07-18 21:40:15,469] DEBUG chain=mesosphere.marathon.api.LeaderProxyFilter-329548d0->mesosphere.marathon.api.LimitConcurrentRequestsFilter-49d30c6f->mesosphere.marathon.api.CORSFilter-1115433e->mesosphere.marathon.api.CacheDisablingFilter-4fdca00a->org.glassfish.jersey.servlet.ServletContainer-5d1e0fbb@5dcc1567==org.glassfish.jersey.servlet.ServletContainer,jsp=null,order=-1,inst=true (org.eclipse.jetty.servlet.ServletHandler:qtp2094310615-27)
[2020-07-18 21:40:15,473] DEBUG call filter mesosphere.marathon.api.LeaderProxyFilter-329548d0 (org.eclipse.jetty.servlet.ServletHandler:qtp2094310615-27)
[2020-07-18 21:40:15,479] DEBUG call filter mesosphere.marathon.api.LimitConcurrentRequestsFilter-49d30c6f (org.eclipse.jetty.servlet.ServletHandler:qtp2094310615-27)
[2020-07-18 21:40:15,480] DEBUG call filter mesosphere.marathon.api.CORSFilter-1115433e (org.eclipse.jetty.servlet.ServletHandler:qtp2094310615-27)
[2020-07-18 21:40:15,490] DEBUG call filter mesosphere.marathon.api.CacheDisablingFilter-4fdca00a (org.eclipse.jetty.servlet.ServletHandler:qtp2094310615-27)
[2020-07-18 21:40:15,491] DEBUG call servlet org.glassfish.jersey.servlet.ServletContainer-5d1e0fbb@5dcc1567==org.glassfish.jersey.servlet.ServletContainer,jsp=null,order=-1,inst=true (org.eclipse.jetty.servlet.ServletHandler:qtp2094310615-27)
[2020-07-18 21:40:15,551] DEBUG Checking for overdue tasks (mesosphere.marathon.core.task.jobs.impl.OverdueInstancesActor$Support:marathon-akka.actor.default-dispatcher-8)
[2020-07-18 21:40:15,606] DEBUG Got ping response for sessionid: 0x17362f333900011 after 1ms (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
[2020-07-18 21:40:15,648] DEBUG startAsync HttpChannelState@29476a6{s=DISPATCHED a=NOT_ASYNC i=true r=IDLE w=false} (org.eclipse.jetty.server.HttpChannelState:qtp2094310615-27)
[2020-07-18 21:40:15,696] DEBUG handled=true async=true committed=false on HttpChannelOverHttp@5cb30625{r=1,c=false,a=DISPATCHED,uri=//test2.local:7070/v2/info} (org.eclipse.jetty.server.Server:qtp2094310615-27)
[2020-07-18 21:40:15,697] DEBUG unhandle HttpChannelState@29476a6{s=DISPATCHED a=STARTED i=true r=IDLE w=false} (org.eclipse.jetty.server.HttpChannelState:qtp2094310615-27)
[2020-07-18 21:40:15,698] DEBUG HttpChannelOverHttp@5cb30625{r=1,c=false,a=ASYNC_WAIT,uri=//test2.local:7070/v2/info} action WAIT (org.eclipse.jetty.server.HttpChannel:qtp2094310615-27)
[2020-07-18 21:40:15,699] DEBUG HttpChannelOverHttp@5cb30625{r=1,c=false,a=ASYNC_WAIT,uri=//test2.local:7070/v2/info} handle exit, result WAIT (org.eclipse.jetty.server.HttpChannel:qtp2094310615-27)
[2020-07-18 21:40:15,701] DEBUG HttpConnection@475cea3e[p=HttpParser{s=END,0 of -1},g=HttpGenerator@4d2dab43{s=START}]=>HttpChannelOverHttp@5cb30625{r=1,c=false,a=ASYNC_WAIT,uri=//test2.local:7070/v2/info}<-SocketChannelEndPoint@1afde438{/192.168.10.219:57323<->/192.168.10.22:7070,OPEN,fill=-,flush=-,to=305/30000}{io=0/0,kio=0,kro=1}->HttpConnection@475cea3e[p=HttpParser{s=END,0 of -1},g=HttpGenerator@4d2dab43{s=START}]=>HttpChannelOverHttp@5cb30625{r=1,c=false,a=ASYNC_WAIT,uri=//test2.local:7070/v2/info} onFillable exit HttpChannelState@29476a6{s=ASYNC_WAIT a=STARTED i=false r=IDLE w=false} null (org.eclipse.jetty.server.HttpConnection:qtp2094310615-27)
[2020-07-18 21:40:15,701] DEBUG ran org.eclipse.jetty.io.ManagedSelector$$Lambda$1423/1947349460@43b0a008 (org.eclipse.jetty.util.thread.QueuedThreadPool:qtp2094310615-27)
[2020-07-18 21:40:16,550] DEBUG Current State: LaunchTokens:100 OffersWanted:false Matchers:0 OfferQueues:0 UnprocessedOffers:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-3)
[2020-07-18 21:40:16,550] DEBUG Reject Elapsed offers. Unprocessed: 0 Overdue:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-3)
[2020-07-18 21:40:18,941] DEBUG Got ping response for sessionid: 0x17362f333900011 after 1ms (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
[2020-07-18 21:40:19,551] DEBUG Current State: LaunchTokens:100 OffersWanted:false Matchers:0 OfferQueues:0 UnprocessedOffers:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-4)
[2020-07-18 21:40:19,551] DEBUG Reject Elapsed offers. Unprocessed: 0 Overdue:0 (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-4)
[2020-07-18 21:40:20,560] DEBUG Checking for overdue tasks (mesosphere.marathon.core.task.jobs.impl.OverdueInstancesActor$Support:marathon-akka.actor.default-dispatcher-3)
[2020-07-18 21:40:22,276] DEBUG Got ping response for sessionid: 0x17362f333900011 after 1ms (org.apache.zookeeper.ClientCnxn:main-SendThread(m01.local:2181))
1.10.25 the request ends with these lines
[2020-07-18 21:40:15,701] DEBUG HttpConnection@475cea3e[p=HttpParser{s=END,0 of -1},g=HttpGenerator@4d2dab43{s=START}]=>HttpChannelOverHttp@5cb30625{r=1,c=false,a=ASYNC_WAIT,uri=//test2.local:7070/v2/info}<-SocketChannelEndPoint@1afde438{/192.168.10.219:57323<->/192.168.10.22:7070,OPEN,fill=-,flush=-,to=305/30000}{io=0/0,kio=0,kro=1}->HttpConnection@475cea3e[p=HttpParser{s=END,0 of -1},g=HttpGenerator@4d2dab43{s=START}]=>HttpChannelOverHttp@5cb30625{r=1,c=false,a=ASYNC_WAIT,uri=//test2.local:7070/v2/info} onFillable exit HttpChannelState@29476a6{s=ASYNC_WAIT a=STARTED i=false r=IDLE w=false} null (org.eclipse.jetty.server.HttpConnection:qtp2094310615-27)
[2020-07-18 21:40:15,701] DEBUG ran org.eclipse.jetty.io.ManagedSelector$$Lambda$1423/1947349460@43b0a008 (org.eclipse.jetty.util.thread.QueuedThreadPool:qtp2094310615-27)
where 1.7.236 continues like this
[info] [2020-07-18 23:00:22,598] DEBUG HttpConnection@df1bc7a[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7caf188e{s=START}]=>HttpChannelOverHttp@1eecfa5c{r=1,c=false,a=ASYNC_WAIT,uri=//test2.local:7071/v2/info}<-SocketChannelEndPoint@3d798ac0{/192.168.10.219:57151<->/192.168.10.22:7071,OPEN,fill=-,flush=-,to=48/30000}{io=0/0,kio=0,kro=1}->HttpConnection@df1bc7a[p=HttpParser{s=END,0 of -1},g=HttpGenerator@7caf188e{s=START}]=>HttpChannelOverHttp@1eecfa5c{r=1,c=false,a=ASYNC_WAIT,uri=//test2.local:7071/v2/info} onFillable exit HttpChannelState@6c10396e{s=ASYNC_WAIT a=STARTED i=false r=IDLE w=false} null (org.eclipse.jetty.server.HttpConnection:qtp1640832113-32)
[info] [2020-07-18 23:00:22,598] DEBUG ReservedThreadExecutor@298e6c95{s=1/1,p=0}@SelectorManager@ServerConnector@75ae4a1f{HTTP/1.1,[http/1.1]}{0.0.0.0:7071}@40fe012 size 1 > capacity (org.eclipse.jetty.util.thread.ReservedThreadExecutor:qtp1640832113-32)
[info] [2020-07-18 23:00:22,598] DEBUG ran ReservedThreadExecutor@298e6c95{s=1/1,p=0}@SelectorManager@ServerConnector@75ae4a1f{HTTP/1.1,[http/1.1]}{0.0.0.0:7071}@40fe012 (org.eclipse.jetty.util.thread.QueuedThreadPool:qtp1640832113-32)
[info] [2020-07-18 23:00:22,600] DEBUG org.eclipse.jetty.server.handler.gzip.GzipHttpOutputInterceptor@262a7e31 compressing java.util.zip.Deflater@5f15d197 (org.eclipse.jetty.server.handler.gzip.GzipHttpOutputInterceptor:scala-execution-context-global-235)
[info] [2020-07-18 23:00:22,600] DEBUG sendResponse info=null content=HeapByteBuffer@2c81139b[p=0,l=755,c=32768,r=755]={<<<\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x00}UM\x8f\x9b0\x10...|\xBbM\xDf\xDf\xFf\x03\x07\xD7aL\x96\x06\x00\x00>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} complete=true committing=true callback=GzipBufferCB@7e0724e3 (org.eclipse.jetty.server.HttpChannel:scala-execution-context-global-235)
[info] [2020-07-18 23:00:22,600] DEBUG COMMIT for /v2/info on HttpChannelOverHttp@1eecfa5c{r=1,c=true,a=ASYNC_WAIT,uri=//test2.local:7071/v2/info}
[info] 200 OK HTTP/1.1
[info] Date: Sat, 18 Jul 2020 21:00:22 GMT
[info] X-Marathon-Leader: http://test2.local:7071
[info] X-Frame-Options: DENY
[info] X-XSS-Protection: 1; mode=block
[info] Cache-Control: no-cache, no-store, must-revalidate
[info] Pragma: no-cache
[info] Expires: 0
[info] Content-Type: application/json
[info] Vary: Accept-Encoding, User-Agent
[info] Content-Encoding: gzip