Uploaded image for project: 'Marathon'
  1. Marathon
  2. MARATHON-8160

Document: Rolling back deployments allows duplicate service ports

    Details

      Description

      Overview

      Marathon enforces uniqueness for service ports; however, the constraint only considers the current target state, and not the former state for some ongoing deployment. Furthermore, rolling back a deployment, understandably, does not trigger validation. As such, it is possible to put Marathon in an invalid state, where 2 apps share the same servicePort.

      Marathon tries to assign unique service ports when servicePort: 0 is provided, but when assigning, it only considers the current target state for some ongoing deployment. This means that when we roll back a deployment it is possible for a unique service-port assignment to no longer be unique.

      Further, since there is no global validation for service port uniqueness, one can simply post the two app definitions with the same service ports, and Marathon will accept them.

      How to reproduce by auto-assignment and rolling back a deployment

      $ http :8080/v2/apps <<-EOF
      
      {
        "id": "/sleeper",
        "container": {
          "docker": {
            "image": "alpine",
            "network": "BRIDGE",
            "portMappings": [
              {
                "containerPort": 80,
                "hostPort": 0,
                "labels": {},
                "name": "default",
                "protocol": "tcp",
                "servicePort": 10000
              }
            ]
          }
        },
        "cmd": "sleep 3600",
        "instances": 1,
        "cpus": 0.05,
        "mem": 128
      }
      EOF
      
      # wait for deployment to finish
      
      $ http PUT :8080/v2/apps/sleeper <<-EOF
      {
        "id": "/sleeper",
        "container": {
          "docker": {
            "image": "alpine",
            "network": "BRIDGE",
            "portMappings": [
            ]
          }
        },
        "cmd": "sleep 3600",
        "instances": 1,
        "cpus": 300,
        "mem": 128
      }
      EOF
      
      # it will never finish; take note of the deployment ID
      
      $ http :8080/v2/apps <<-EOF
      {
        "id": "/sleeper-2",
        "container": {
          "docker": {
            "image": "alpine",
            "network": "BRIDGE",
            "portMappings": [
              {
                "containerPort": 80,
                "hostPort": 0,
                "labels": {},
                "name": "default",
                "protocol": "tcp",
                "servicePort": 10000
              }
            ]
          }
        },
        "cmd": "sleep 3600",
        "instances": 1,
        "cpus": 0.1,
        "mem": 128
      }
      EOF
      
      # Finally, roll back the deployment (with the deployment ID returned by the 2nd, never-ending step)
      
      $ http DELETE :8080/v2/deployments/da73f994-b0e8-4e87-830b-20456fc59d76
      
      # Let's see our service ports
      
      $ http :8080/v2/tasks "Accept: text/plain"
      ...
      sleeper	10000	bastion-wifi.lan:31839
      sleeper-2	10000	bastion-wifi.lan:31339
      

      How to reproduce by posting two app definitions with the same service ports:

      
      $ http :8080/v2/apps <<-EOF
      {
        "id": "/sleeper",
        "container": {
          "docker": {
            "image": "alpine"
          }
        },
        "networks": [{"mode": "host"}],
        "portDefinitions": [{"port": 10000, "protocol": "tcp"}, {"port": 10001, "protocol": "tcp"}],
        "requirePorts": false,
        "cmd": "sleep 3600",
        "instances": 1,
        "cpus": 0.05,
        "mem": 128
      }
      EOF
      
      
      $ http :8080/v2/apps <<-EOF
      {
        "id": "/sleeper-2",
        "container": {
          "docker": {
            "image": "alpine"
          }
        },
        "networks": [{"mode": "host"}],
        "portDefinitions": [{"port": 10000, "protocol": "tcp"}, {"port": 10001, "protocol": "tcp"}],
        "requirePorts": false,
        "cmd": "sleep 3600",
        "instances": 1,
        "cpus": 0.05,
        "mem": 128
      }
      EOF
      
      $ http :8080/v2/tasks "Accept: text/plain"
      
      
      sleeper	10000	bastion-wifi.lan:31501
      sleeper	10001	bastion-wifi.lan:31502
      sleeper-2	10000	bastion-wifi.lan:31780
      sleeper-2	10001	bastion-wifi.lan:31781
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                tharper Tim Harper
                Team:
                Orchestration Team
                Watchers:
                daltonmatos, Nikita Melkozerov (Inactive), Tim Harper
              • Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: