Uploaded image for project: 'Marathon'
  1. Marathon
  2. MARATHON-4758

Upgrade may stuck if app new launched task FINISH before health check passing

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Low
    • Resolution: Won't Do
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Deployments
    • Labels:

      Description

      Marathon Version: 1.3.6
      Step to reproduce:
      1. start a simple web server app with health check, with 1 instance, wait unit running

      {
        "id": "/test",
        "cmd": null,
        "cpus": 1,
        "mem": 128,
        "instances": 1,
        "container": {
          "type": "DOCKER",
          "volumes": [],
          "docker": {
            "image": "httpd:2.4",
            "network": "BRIDGE",
            "portMappings": [
              {
                "containerPort": 80,
                "protocol": "tcp"
              }
            ]
          }
        },
        "healthChecks": [
          {
            "path": "/",
            "protocol": "HTTP",
            "portIndex": 0,
            "gracePeriodSeconds": 60,
            "intervalSeconds": 20,
            "timeoutSeconds": 20,
            "maxConsecutiveFailures": 3,
            "ignoreHttp1xx": false
          }
        ]
      }
      

      2. modify app configuration, change app cmd to "sleep 20", and upgrade

      Expected result:
      1. Newly launched tasks fail health check
      2. Newly launched task finish
      3. Marathon launch another task after the task finish
      4. repeat step 1

      Actual result:
      1. Newly launched tasks fail health check
      2. Newly launched task finish
      3. From now on, marathon deployment get stuck and do not try to launch new task

      I think this affects marathon 1.3.x and 1.1.x

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              GitHub_fengyehong fengyehong (Inactive)
              Team:
              Orchestration Team
              Watchers:
              Aleksey Dukhovniy, Jason Gilanfarr (Inactive)
            • Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: