Uploaded image for project: 'Marathon'
  1. Marathon
  2. MARATHON-1730

Applications with persistent local volumes do not start

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Medium
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: Marathon 1.4.7
    • Component/s: None
    • Labels:

      Description

      Adding an application with persistent local volumes does not start on marathon. I have tried multiple different app definitions. Logs and definitions are included below.

      • statefull-application-working is a python tool.
      • stateful-volumes-working is a simple application with a local volume associated.
      • volume is a very simple application which was created with the ui.

      Logs

      [2016-04-05 13:06:35,274] INFO Processing ForceExpunge(task [stateful-application.49cafa27-fb2c-11e5-9ba8-8a2561d6480a]) for task [stateful-application.49cafa27-fb2c-11e5-9ba8-8a2561d6480a] (mesosphere.marathon.core.launcher.impl.OfferProcessorImpl:ForkJoinPool-2-worker-1)
      [2016-04-05 13:06:35,274] INFO Ignoring ForceExpunge for [task [stateful-application.49cafa27-fb2c-11e5-9ba8-8a2561d6480a]], task does not exist (mesosphere.marathon.core.task.tracker.impl.TaskOpProcessorImpl$TaskStateOpResolver:marathon-akka.actor.default-dispatcher-8)
      [2016-04-05 13:06:35,274] INFO accepted unreserveAndDestroy for task [stateful-application.49cafa27-fb2c-11e5-9ba8-8a2561d6480a] in offer [9d8f8a58-ef25-4abc-adf0-78e046672dfa-O1421] (mesosphere.marathon.core.matcher.reconcile.impl.OfferMatcherReconciler:ForkJoinPool-2-worker-1)
      [2016-04-05 13:06:36,275] INFO Processing ForceExpunge(task [stateful-application.49cafa27-fb2c-11e5-9ba8-8a2561d6480a]) for task [stateful-application.49cafa27-fb2c-11e5-9ba8-8a2561d6480a] (mesosphere.marathon.core.launcher.impl.OfferProcessorImpl:ForkJoinPool-2-worker-7)
      [2016-04-05 13:06:36,275] INFO Ignoring ForceExpunge for [task [stateful-application.49cafa27-fb2c-11e5-9ba8-8a2561d6480a]], task does not exist (mesosphere.marathon.core.task.tracker.impl.TaskOpProcessorImpl$TaskStateOpResolver:marathon-akka.actor.default-dispatcher-12)
      [2016-04-05 13:06:36,275] INFO accepted unreserveAndDestroy for task [stateful-application.49cafa27-fb2c-11e5-9ba8-8a2561d6480a] in offer [9d8f8a58-ef25-4abc-adf0-78e046672dfa-O1422] (mesosphere.marathon.core.matcher.reconcile.impl.OfferMatcherReconciler:ForkJoinPool-2-worker-9)
      [2016-04-05 13:06:37,276] INFO Processing ForceExpunge(task [stateful-application.49cafa27-fb2c-11e5-9ba8-8a2561d6480a]) for task [stateful-application.49cafa27-fb2c-11e5-9ba8-8a2561d6480a] (mesosphere.marathon.core.launcher.impl.OfferProcessorImpl:ForkJoinPool-2-worker-1)
      [2016-04-05 13:06:37,276] INFO Ignoring ForceExpunge for [task [stateful-application.49cafa27-fb2c-11e5-9ba8-8a2561d6480a]], task does not exist (mesosphere.marathon.core.task.tracker.impl.TaskOpProcessorImpl$TaskStateOpResolver:marathon-akka.actor.default-dispatcher-12)
      [2016-04-05 13:06:37,276] INFO accepted unreserveAndDestroy for task [stateful-application.49cafa27-fb2c-11e5-9ba8-8a2561d6480a] in offer [9d8f8a58-ef25-4abc-adf0-78e046672dfa-O1423] (mesosphere.marathon.core.matcher.reconcile.impl.OfferMatcherReconciler:ForkJoinPool-2-worker-5)
      [2016-04-05 13:06:37,294] INFO initiate task reconciliation (mesosphere.marathon.MarathonSchedulerActor:marathon-akka.actor.default-dispatcher-7)
      [2016-04-05 13:06:37,294] INFO reconcile [/stateful-application-working] with latest version [2016-04-05T12:49:00.912Z] (mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$5c8ec9ec:ForkJoinPool-2-worker-11)
      [2016-04-05 13:06:37,294] INFO reconcile [/local-volumes-working] with latest version [2016-04-05T12:50:43.341Z] (mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$5c8ec9ec:ForkJoinPool-2-worker-1)
      [2016-04-05 13:06:37,294] INFO reconcile [/volumes] with latest version [2016-04-05T12:55:40.039Z] (mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$5c8ec9ec:ForkJoinPool-2-worker-7)
      [2016-04-05 13:06:37,294] INFO Requesting task reconciliation with the Mesos master (mesosphere.marathon.SchedulerActions:ForkJoinPool-2-worker-15)
      [2016-04-05 13:06:37,294] INFO addAllFor [/volumes] version [2016-04-05T12:55:40.039Z] (mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$5c8ec9ec:ForkJoinPool-2-worker-7)
      [2016-04-05 13:06:37,294] INFO addAllFor [/local-volumes-working] version [2016-04-05T12:50:43.341Z] (mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$5c8ec9ec:ForkJoinPool-2-worker-1)
      [2016-04-05 13:06:37,294] INFO task reconciliation has finished (mesosphere.marathon.MarathonSchedulerActor:marathon-akka.actor.default-dispatcher-12)
      [2016-04-05 13:06:37,294] INFO addAllFor [/stateful-application-working] version [2016-04-05T12:49:00.912Z] (mesosphere.marathon.health.MarathonHealthCheckManager$$EnhancerByGuice$$5c8ec9ec:ForkJoinPool-2-worker-11)
      [2016-04-05 13:06:38,277] INFO Processing ForceExpunge(task [stateful-application.49cafa27-fb2c-11e5-9ba8-8a2561d6480a]) for task [stateful-application.49cafa27-fb2c-11e5-9ba8-8a2561d6480a] (mesosphere.marathon.core.launcher.impl.OfferProcessorImpl:ForkJoinPool-2-worker-9)
      [2016-04-05 13:06:38,277] INFO Ignoring ForceExpunge for [task [stateful-application.49cafa27-fb2c-11e5-9ba8-8a2561d6480a]], task does not exist (mesosphere.marathon.core.task.tracker.impl.TaskOpProcessorImpl$TaskStateOpResolver:marathon-akka.actor.default-dispatcher-12)
      [2016-04-05 13:06:38,277] INFO accepted unreserveAndDestroy for task [stateful-application.49cafa27-fb2c-11e5-9ba8-8a2561d6480a] in offer [9d8f8a58-ef25-4abc-adf0-78e046672dfa-O1424] (mesosphere.marathon.core.matcher.reconcile.impl.OfferMatcherReconciler:ForkJoinPool-2-worker-9)
      

      App definitions:

       JSON
      {
        "apps": [
          {
            "id": "/stateful-application-working",
            "cmd": "pip install markdoc && markdoc init ./home || echo && echo \"# Started At `date`\" >> ./home/wiki/index.md && cd ./home && markdoc build && (markdoc serve -i 0.0.0.0 -p $PORT0 & while true; do sleep 5; echo `date` >> wiki/index.md; echo >> wiki/index.md; markdoc build; done)",
            "instances": 1,
            "cpus": 0.1,
            "mem": 32,
            "disk": 0,
            "ports": [
              0
            ],
            "maxLaunchDelaySeconds": 3600,
      
            "container": {
              "type": "MESOS",
              "volumes": [
                {
                  "containerPath": "home",
                  "mode": "RW",
                  "persistent": {
                    "size": 100
                  }
                }
              ]
            },
            "upgradeStrategy": {
              "minimumHealthCapacity": 0.5,
              "maximumOverCapacity": 0
            },
            "residency": {
              "relaunchEscalationTimeoutSeconds": 10,
              "taskLostBehavior": "WAIT_FOREVER"
            }
          },
          {
            "id": "/local-volumes-working",
            "cmd": "sleep 1000",
            "instances": 1,
            "cpus": 0.1,
            "mem": 32,
            "disk": 0,
            "ports": [
              0
            ],
            "container": {
              "type": "MESOS",
              "volumes": [
                {
                  "containerPath": "home",
                  "mode": "RW",
                  "persistent": {
                    "size": 100
                  }
                }
              ]
            },
            "upgradeStrategy": {
              "minimumHealthCapacity": 0.5,
              "maximumOverCapacity": 0
            },
            "residency": {
              "relaunchEscalationTimeoutSeconds": 10,
              "taskLostBehavior": "WAIT_FOREVER"
            },
            "versionInfo": {
              "lastScalingAt": "2016-04-05T12:50:43.341Z",
              "lastConfigChangeAt": "2016-04-05T12:50:43.341Z"
            }
          },
          {
            "id": "/volumes",
            "cmd": "sleep 1000;",
            "instances": 1,
            "cpus": 0.1,
            "mem": 128,
            "disk": 0,
            "container": {
              "type": "MESOS",
              "volumes": [
                {
                  "containerPath": "home",
                  "mode": "RW",
                  "persistent": {
                    "size": 1024
                  }
                }
              ]
            },
            "upgradeStrategy": {
              "minimumHealthCapacity": 0.5,
              "maximumOverCapacity": 0
            },
            "residency": {
              "relaunchEscalationTimeoutSeconds": 10,
              "taskLostBehavior": "WAIT_FOREVER"
            }
          }
        ]
      }
      
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              philipph Philipp Hinrichsen
              Team:
              Orchestration Team
              Watchers:
            • Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: