Uploaded image for project: 'Marathon'
  1. Marathon
  2. MARATHON-8070

Marathon should inform why it can not start

    Details

    • Type: Task
    • Status: Resolved
    • Priority: Medium
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Leader Election
    • Labels:

      Description

      Trying to run Marathon as it described in README fails.

      $ git rev-parse HEAD                                                 
      67adf0290b0f0aecfbf465f5647b4a699a4e8afc
      
      $ sbt 'run --master localhost:5050 --zk zk://localhost:2181/marathon'
      [...]
      [info] [2018-02-07 14:29:37,935] INFO  Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper:JMX exporting thread)
      [info] [2018-02-07 14:29:37,935] INFO  Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper:JMX exporting thread)
      [info] [2018-02-07 14:29:37,935] INFO  Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper:JMX exporting thread)
      [info] [2018-02-07 14:29:37,935] INFO  Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper:JMX exporting thread)
      [info] [2018-02-07 14:29:37,935] INFO  Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper:JMX exporting thread)
      [info] [2018-02-07 14:29:37,935] INFO  Client environment:os.version=4.13.0-32-generic (org.apache.zookeeper.ZooKeeper:JMX exporting thread)
      [info] [2018-02-07 14:29:37,935] INFO  Client environment:user.name=tomasz.janiszewski (org.apache.zookeeper.ZooKeeper:JMX exporting thread)
      [info] [2018-02-07 14:29:37,935] INFO  Client environment:user.home=/home/local/ALLEGROGROUP/tomasz.janiszewski (org.apache.zookeeper.ZooKeeper:JMX exporting thread)
      [info] [2018-02-07 14:29:37,935] INFO  Client environment:user.dir=/home/local/ALLEGROGROUP/tomasz.janiszewski/workspace/src/github.com/mesosphere/marathon (org.apache.zookeeper.ZooKeeper:JMX exporting thread)
      [info] [2018-02-07 14:29:37,936] INFO  Initiating client connection, connectString=localhost:2181 sessionTimeout=10000 watcher=org.apache.curator.ConnectionState@7378c4a4 (org.apache.zookeeper.ZooKeeper:JMX exporting thread)
      [info] [2018-02-07 14:29:37,959] INFO  Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn:JMX exporting thread-SendThread(localhost:2181))
      [info] [2018-02-07 14:29:37,969] INFO  Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session (org.apache.zookeeper.ClientCnxn:JMX exporting thread-SendThread(localhost:2181))
      [info] [2018-02-07 14:29:37,973] INFO  Default schema (org.apache.curator.framework.imps.CuratorFrameworkImpl:JMX exporting thread)
      [info] [2018-02-07 14:29:37,985] INFO  Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x1616f4809340008, negotiated timeout = 10000 (org.apache.zookeeper.ClientCnxn:JMX exporting thread-SendThread(localhost:2181))
      [info] [2018-02-07 14:29:37,998] INFO  State change: CONNECTED (org.apache.curator.framework.state.ConnectionStateManager:JMX exporting thread-EventThread)
      [info] [2018-02-07 14:29:38,592] INFO  Started TaskTrackerUpdateStepsProcessorImpl with steps:
      [info] * continueOnError(notifyHealthCheckManager)
      [info] * continueOnError(notifyRateLimiter)
      [info] * continueOnError(notifyLaunchQueue)
      [info] * continueOnError(postTaskStatusEvent)
      [info] * continueOnError(scaleApp) (mesosphere.marathon.core.task.tracker.impl.InstanceTrackerUpdateStepProcessorImpl:JMX exporting thread)
      [info] [2018-02-07 14:29:38,715] INFO  Calling reviveOffers is enabled. Use --disable_revive_offers_for_new_apps to disable. (mesosphere.marathon.core.flow.FlowModule:JMX exporting thread)
      [info] [2018-02-07 14:29:38,877] INFO  Loading plugins implementing 'mesosphere.marathon.plugin.task.RunSpecTaskProcessor' from these urls: [] (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:JMX exporting thread)
      [info] [2018-02-07 14:29:38,884] INFO  Found 0 plugins. (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:JMX exporting thread)
      [info] [2018-02-07 14:29:38,885] INFO  Loading plugins implementing 'mesosphere.marathon.plugin.validation.RunSpecValidator' from these urls: [] (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:JMX exporting thread)
      [info] [2018-02-07 14:29:38,886] INFO  Found 0 plugins. (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:JMX exporting thread)
      [info] [2018-02-07 14:29:39,044] INFO  Will do leader election through localhost:2181 (mesosphere.marathon.core.election.CuratorElectionStream$:JMX exporting thread)
      [info] [2018-02-07 14:29:39,048] INFO  Starting (org.apache.curator.framework.imps.CuratorFrameworkImpl:JMX exporting thread)
      [info] [2018-02-07 14:29:39,048] INFO  Initiating client connection, connectString=localhost:2181 sessionTimeout=10000 watcher=org.apache.curator.ConnectionState@524dd373 (org.apache.zookeeper.ZooKeeper:JMX exporting thread)
      [info] [2018-02-07 14:29:39,050] INFO  Default schema (org.apache.curator.framework.imps.CuratorFrameworkImpl:JMX exporting thread)
      [info] [2018-02-07 14:29:39,051] INFO  Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn:JMX exporting thread-SendThread(localhost:2181))
      [info] [2018-02-07 14:29:39,051] INFO  Socket connection established to localhost/127.0.0.1:2181, initiating session (org.apache.zookeeper.ClientCnxn:JMX exporting thread-SendThread(localhost:2181))
      [info] [2018-02-07 14:29:39,060] INFO  Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x1616f4809340009, negotiated timeout = 10000 (org.apache.zookeeper.ClientCnxn:JMX exporting thread-SendThread(localhost:2181))
      [info] [2018-02-07 14:29:39,060] INFO  State change: CONNECTED (org.apache.curator.framework.state.ConnectionStateManager:JMX exporting thread-EventThread)
      [info] [2018-02-07 14:29:39,116] INFO  Loading plugins implementing 'mesosphere.marathon.plugin.auth.Authenticator' from these urls: [] (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:JMX exporting thread)
      [info] [2018-02-07 14:29:39,117] INFO  Found 0 plugins. (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:JMX exporting thread)
      [info] [2018-02-07 14:29:39,119] INFO  Loading plugins implementing 'mesosphere.marathon.plugin.auth.Authorizer' from these urls: [] (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:JMX exporting thread)
      [info] [2018-02-07 14:29:39,119] INFO  Found 0 plugins. (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:JMX exporting thread)
      [info] [2018-02-07 14:29:39,131] INFO  Started status update processor (mesosphere.marathon.core.task.update.impl.TaskStatusUpdateProcessorImpl:JMX exporting thread)
      [info] [2018-02-07 14:29:39,362] INFO  Loading plugins implementing 'mesosphere.marathon.plugin.scheduler.SchedulerPlugin' from these urls: [] (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:JMX exporting thread)
      [info] [2018-02-07 14:29:39,363] INFO  Found 0 plugins. (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:JMX exporting thread)
      [info] [2018-02-07 14:29:39,435] INFO  All actors suspended:
      [info] * Actor[akka://marathon/user/rateLimiter#-467286095]
      [info] * Actor[akka://marathon/user/offerMatcherLaunchTokens#613574621]
      [info] * Actor[akka://marathon/user/instanceTracker#1932475502]
      [info] * Actor[akka://marathon/user/deploymentManager#118952040]
      [info] * Actor[akka://marathon/user/expungeOverdueLostTasks#-1489549880]
      [info] * Actor[akka://marathon/user/reviveOffersWhenWanted#-1888869131]
      [info] * Actor[akka://marathon/user/offerMatcherStatistics#1037048989]
      [info] * Actor[akka://marathon/user/launchQueue#2114338652]
      [info] * Actor[akka://marathon/user/offersWantedForReconciliation#-37172566]
      [info] * Actor[akka://marathon/user/killOverdueStagedTasks#1675882077]
      [info] * Actor[akka://marathon/user/taskKillServiceActor#-581449634]
      [info] * Actor[akka://marathon/user/offerMatcherManager#2004724278] (mesosphere.marathon.core.leadership.impl.LeadershipCoordinatorActor:marathon-akka.actor.default-dispatcher-4)
      [info] [2018-02-07 14:29:39,529] INFO  Logging initialized @8914ms (org.eclipse.jetty.util.log:JMX exporting thread)
      [info] [2018-02-07 14:29:39,589] INFO  Adding HTTP support. (mesosphere.chaos.http.HttpModule:JMX exporting thread)
      [info] [2018-02-07 14:29:39,590] INFO  No HTTPS support configured. (mesosphere.chaos.http.HttpModule:JMX exporting thread)
      [info] [2018-02-07 14:29:39,658] INFO  Starting up (mesosphere.marathon.MarathonSchedulerService:MarathonSchedulerService)
      [info] [2018-02-07 14:29:39,658] INFO  jetty-9.3.6.v20151106 (org.eclipse.jetty.server.Server:$anon$1 STARTING)
      [info] [2018-02-07 14:29:39,670] INFO  Beginning run (mesosphere.marathon.MarathonSchedulerService:MarathonSchedulerService)
      [info] [2018-02-07 14:29:39,711] INFO  starting leader latch (mesosphere.marathon.core.election.CuratorElectionStream$:MarathonSchedulerService)
      [info] [2018-02-07 14:29:39,781] INFO  Now standing by. Closing existing handles and rejecting new. (mesosphere.marathon.core.event.impl.stream.HttpEventStreamActor:marathon-akka.actor.default-dispatcher-7)
      [info] [2018-02-07 14:29:39,862] INFO  Leader won: polpc03568:8080 (mesosphere.marathon.core.election.CuratorElectionStream$:marathon-akka.actor.default-dispatcher-7)
      [info] [2018-02-07 14:29:39,872] INFO  As new leader running the driver (mesosphere.marathon.MarathonSchedulerService:pool-1-thread-1)
      [info] [2018-02-07 14:29:40,057] INFO  Registering com.codahale.metrics.jersey.InstrumentedResourceMethodDispatchAdapter as a provider class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:$anon$1 STARTING)
      [info] [2018-02-07 14:29:40,060] INFO  Registering mesosphere.marathon.api.MarathonExceptionMapper as a provider class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:$anon$1 STARTING)
      [info] [2018-02-07 14:29:40,061] INFO  Registering mesosphere.marathon.api.SystemResource as a root resource class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:$anon$1 STARTING)
      [info] [2018-02-07 14:29:40,061] INFO  Registering mesosphere.marathon.api.v2.AppsResource as a root resource class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:$anon$1 STARTING)
      [info] [2018-02-07 14:29:40,062] INFO  Registering mesosphere.marathon.api.v2.PodsResource as a root resource class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:$anon$1 STARTING)
      [info] [2018-02-07 14:29:40,062] INFO  Registering mesosphere.marathon.api.v2.TasksResource as a root resource class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:$anon$1 STARTING)
      [info] [2018-02-07 14:29:40,062] INFO  Registering mesosphere.marathon.api.v2.QueueResource as a root resource class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:$anon$1 STARTING)
      [info] [2018-02-07 14:29:40,062] INFO  Registering mesosphere.marathon.api.v2.GroupsResource as a root resource class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:$anon$1 STARTING)
      [info] [2018-02-07 14:29:40,062] INFO  Registering mesosphere.marathon.api.v2.InfoResource as a root resource class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:$anon$1 STARTING)
      [info] [2018-02-07 14:29:40,062] INFO  Registering mesosphere.marathon.api.v2.LeaderResource as a root resource class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:$anon$1 STARTING)
      [info] [2018-02-07 14:29:40,063] INFO  Registering mesosphere.marathon.api.v2.DeploymentsResource as a root resource class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:$anon$1 STARTING)
      [info] [2018-02-07 14:29:40,063] INFO  Registering mesosphere.marathon.api.v2.SchemaResource as a root resource class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:$anon$1 STARTING)
      [info] [2018-02-07 14:29:40,063] INFO  Registering mesosphere.marathon.api.v2.PluginsResource as a root resource class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:$anon$1 STARTING)
      [info] [2018-02-07 14:29:40,066] INFO  Initiating Jersey application, version 'Jersey: 1.18.6 10/26/2016 11:30 AM' (com.sun.jersey.server.impl.application.WebApplicationImpl:$anon$1 STARTING)
      [info] [2018-02-07 14:29:40,179] INFO  Binding com.codahale.metrics.jersey.InstrumentedResourceMethodDispatchAdapter to GuiceManagedComponentProvider with the scope "Singleton" (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:$anon$1 STARTING)
      [info] [2018-02-07 14:29:40,216] INFO  Binding mesosphere.marathon.api.MarathonExceptionMapper to GuiceManagedComponentProvider with the scope "Singleton" (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:$anon$1 STARTING)
      [info] [2018-02-07 14:29:40,490] INFO  Leadership ended gracefully; exiting (mesosphere.marathon.core.election.ElectionServiceImpl:marathon-akka.actor.default-dispatcher-7)
      [info] [2018-02-07 14:29:40,497] INFO  Closing leader latch (mesosphere.marathon.core.election.CuratorElectionStream$:pool-1-thread-1)
      [info] [2018-02-07 14:29:40,502] INFO  Leader latch closed (mesosphere.marathon.core.election.CuratorElectionStream$:pool-1-thread-1)
      [info] [2018-02-07 14:29:40,525] INFO  Message [akka.dispatch.ContextWrapper] without sender to Actor[akka://marathon/user/StreamSupervisor-1/flow-1-0-map#721332049] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. (akka.actor.RepointableActorRef:marathon-akka.actor.default-dispatcher-2)
      [info] [2018-02-07 14:29:40,533] INFO  Shutting down services (mesosphere.marathon.MarathonApp:shutdownHook3)
      [info] [INFO] [02/07/2018 14:29:40.598] [kamon-akka.actor.default-dispatcher-6] [akka://kamon/user/metrics] Message [akka.dispatch.ContextWrapper] from Actor[akka://kamon/user/datadog-metrics-sender#1732987717] to Actor[akka://kamon/user/metrics#-1663586123] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
      [info] [INFO] [02/07/2018 14:29:40.599] [kamon-akka.actor.default-dispatcher-6] [akka://kamon/user/metrics] Message [akka.dispatch.ContextWrapper] from Actor[akka://marathon/user/$a#-2120643681] to Actor[akka://kamon/user/metrics#-1663586123] was not delivered. [2] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
      [error] java.lang.RuntimeException: Nonzero exit code returned from runner: 137
      [error] 	at sbt.ForkRun.processExitCode$1(Run.scala:33)
      [error] 	at sbt.ForkRun.run(Run.scala:42)
      [error] 	at sbt.Defaults$.$anonfun$bgRunTask$5(Defaults.scala:1168)
      [error] 	at sbt.Defaults$.$anonfun$bgRunTask$5$adapted(Defaults.scala:1163)
      [error] 	at sbt.internal.BackgroundThreadPool.$anonfun$run$1(DefaultBackgroundJobService.scala:366)
      [error] 	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
      [error] 	at scala.util.Try$.apply(Try.scala:209)
      [error] 	at sbt.internal.BackgroundThreadPool$BackgroundRunnable.run(DefaultBackgroundJobService.scala:289)
      [error] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      [error] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      [error] 	at java.lang.Thread.run(Thread.java:748)
      [error] (Compile / run) Nonzero exit code returned from runner: 137
      [error] Total time: 196 s, completed Feb 7, 2018 2:29:42 PM
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                matthias.eichstedt Matthias Eichstedt
                Reporter:
                janisz janisz
                Team:
                Orchestration Team
                Watchers:
                Alena Varkockova, Ivan Chernetsky, janisz, Matthias Eichstedt, Nikita Melkozerov, Tim Harper
              • Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: