Uploaded image for project: 'Marathon'
  1. Marathon
  2. MARATHON-4414

very high minimumHealthCapacity (close to 1) blocks deployment

    Details

      Description

      the problem

      I am trying to do one by one rolling update, but to achieve that i have to match minimumHealthCapacity with the number of instances.
      if I set minimumHealthCapacity too low than more that one instance is killed, if i set it too high the deployment is blocked.

      the theory

      The use case is rolling update on a proxy program that is run on every machine using static port - hence over capacity is no supported.

      An update deployment is blocked due to an impossible upgradeStrategy when these factors are applied:

      • minimumHealthCapacity < 1
      • maximumOverCapacity = 0
      • 1 / instance > 1 - minimumHealthCapacity
        in my case there are 5 slaves so:

        case 1

        config

      minimumHealthCapacity = 0.5
      maximumOverCapacity = 0

      result

      (1 / 5) 0.2 > (1 - 0.5) 0.5 = false
      Deployment work as expected - rolling update two at a time

      • minimumHealthCapacity seem to be rounded up to an actual 0.6

        case 2

        config

      minimumHealthCapacity = 0.8
      maximumOverCapacity = 0

      result

      (1 / 5) 0.2 > (1 - 0.8) 0.2 = false
      Deployment work as expected - rolling update one at a time

      case 3

      config

      minimumHealthCapacity = 0.9
      maximumOverCapacity = 0

      result

      (1 / 5) 0.2 > (1 - 0.9) 0.1 = true
      Deployment is 'waiting' and logs show Insufficient resources for...

      the bottom line (and the suggestion)

      • (Currently) if minimumHealthCapacity = 1 and no over capacity allowed deployment should be blocked because that is in-fact an impossible request
      • (Currently) if minimumHealthCapacity is lower than 1 it should be rounded up to the nearest 1 / instance tick.
      • (Maybe also) if minimumHealthCapacity is lower than 1 BUT rounding it up to the nearest tick equals 1, then it should be corrected to 1 - (1 / instances)
        That would allow setting minimumHealthCapacity to 0.99 - always meaning one at the time updates.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                GitHub_mrbar42 MrBar42 (Inactive)
                Team:
                Orchestration Team
                Watchers:
                Jason Gilanfarr (Inactive)
              • Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: