Uploaded image for project: 'Marathon'
  1. Marathon
  2. MARATHON-8011

Deployment takes too much time when suspending app and changing other fields at the same time

    Details

      Description

      Hello all,

      I observed a behavior with Marathon and I would like to know what do you think about it, if it's a bug of not.

      The behavior affects the suspension of an app. The simplest request to suspend an app is this:

      Assuming an app called /sleep.

      curl -X PUT -d '{"instances": 0}' -H "Content-type: application/json" http://10.168.26.72:8080/v2/apps/sleep
      {"version":"2018-01-10T13:13:25.049Z","deploymentId":"0a8b4fcb-68f8-4601-9fa1-119b6e4f8df7"}
      

      This works as expected. the deployment is correctly created, the Deployments tab on Marathon UI shows it and all app's tasks receive the kill signal almost instantly.

      however, if you change some other fields of the app in the same request, like this:

      curl -X PUT -d '{"instances": 0, "labels": {"some-label": "a-value"}}' -H "Content-type: application/json" http://10.168.26.72:8080/v2/apps/sleep
      {"version":"2018-01-10T13:26:08.164Z","deploymentId":"17af9082-3d8a-4d60-a9c4-526f9988890d"}
      

      Although the API returns a valid DeploymentId, the deployment is not executed, that is, the Deployments tab on the UI does not show any active deploy, the /v2/deployments endpoint returns an empty list and the kill signal does not reach the tasks until some minutes (delays from 1min to 5min during my tests) later.

      I don't know if the delay has any relation to the reconciliation loop that Marathon does.

      Important to note is that this problem only happens if some other field (besides the instances field) changes. If my app already has the label "some-label=some-value" the "problematic" PUT request works as expected.

      Let me know what do you think about this behavior.

      We use Marathon 1.3.13 in production right now and I tested with 1.14.11 locally. Both have this problem. I related this ticket to the 1.4.10 version because the version 1.4.11 seem not be created here in JIRA yet.

      Thanks!

       

        Attachments

          Activity

            People

            • Assignee:
              nikitamelkozerov Nikita Melkozerov
              Reporter:
              daltonmatos daltonmatos
              Team:
              Orchestration Team
              Watchers:
            • Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: