Uploaded image for project: 'Marathon'
  1. Marathon
  2. MARATHON-3307

Authenticated private registry seems to fail with Docker 1.7 auth format?


    • Type: Task
    • Status: Resolved
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:


      Marathon 0.8.2, Mesos 0.22.1, Docker 1.7.0

      I've got a herd of boxes in a mesos + marathon cluster which run Docker 1.7.0 and pull their images from a private registry that uses authentication. I've not had issues with public registries executing the same tasks.

      On each mesos slave, I've actually got login credentials for the private registry in the .docker/config.json file, so I added that file as a URI in the marathon app spec. To wit, if I run docker pull from the CLI of the mesos slave, it pulls just fine.

      When trying to make magic with Marathon, though, I get the errors further below in mesos-slave.ERROR

          "container": {
          "type": "DOCKER",
          "docker": {
            "image": "docker-registry.domain:443/image:latest",
            "forcePullImage": true,
            "network": "BRIDGE",
            "privileged": false,
            "portMappings": [
                "containerPort": 8080,
                "hostPort": 0,
                "servicePort": 8080,
                "protocol": "tcp"
            "parameters": [
              { "key": "env", "value": "ENV=dev" }
        "id": "tomcat",
          "cpus": 0.5,
          "mem": 512.0,
          "instances": 2,
        "uris": [
        "cmd": "$CATALINA_HOME/bin/catalina.sh run",
        "constraints": [
          ["hostname", "UNIQUE"]
        "labels": {
          "environment": "dev",
          "network": "Red"
        "healthChecks": [
            "protocol": "TCP",
            "gracePeriodSeconds": 3,
            "intervalSeconds": 10,
            "timeoutSeconds": 10,
            "maxConsecutiveFailures": 3
        "upgradeStrategy": {
          "minimumHealthCapacity": 0.5,
          "maximumOverCapacity": 0.25

      mesos-slave.ERROR errors

      E0710 05:08:34.904747  6066 slave.cpp:3112] Container 'adc2312f-7072-4f25-bfb1-91f022354713' for executor 'tomcat.b706d5c5-26c1-11e5-bd2e-b6087a30df50' of framework '20150626-194757-3682254739-5050-27187-0000' failed to start: Failed to 'docker pull docker-registry.domain:443/image:latest': exit status = exited with status 1 stderr =
      E0710 05:08:34.906522  6066 slave.cpp:3207] Termination of executor 'tomcat.b706d5c5-26c1-11e5-bd2e-b6087a30df50' of framework '20150626-194757-3682254739-5050-27187-0000' failed: Unknown container: adc2312f-7072-4f25-bfb1-91f022354713
      E0710 05:08:34.907197  6069 slave.cpp:3461] Failed to unmonitor container for executor tomcat.b706d5c5-26c1-11e5-bd2e-b6087a30df50 of framework 20150626-194757-3682254739-5050-27187-0000: Not monitored

      Looking at Mesos' Sandbox stderr for this host and task, this is all that's there (info level stuff):

      I0710 05:19:39.338995 27104 fetcher.cpp:214] Fetching URI 'file://localhost/root/.docker/config.json'
      I0710 05:19:39.339140 27104 fetcher.cpp:194] Copying resource from '/root/.docker/config.json' to '/tmp/mesos/slaves/20150626-194757-3682254739-5050-27187-S11/frameworks/20150626-194757-3682254739-5050-27187-0000/executors/tomcat.43362beb-26c3-11e5-bd2e-b6087a30df50/runs/fc2a6e08-a13e-4c8b-9409-11f380f4830f'
      I0710 05:19:39.343672 27104 fetcher.cpp:329] Skipped extracting path '/tmp/mesos/slaves/20150626-194757-3682254739-5050-27187-S11/frameworks/20150626-194757-3682254739-5050-27187-0000/executors/tomcat.43362beb-26c3-11e5-bd2e-b6087a30df50/runs/fc2a6e08-a13e-4c8b-9409-11f380f4830f/config.json'

      I tried re-formatting config.json into .dockercfg, but the underlying Docker barfed on that saying it's the wrong auth format.

      I admit to being a new user of Marathon; am I missing the painfully obvious or is there legitimately something wrong here? Happy to grab other info if it would help.




            • Assignee:
              GitHub_pdzilla pdzilla (Inactive)
              Orchestration Team
            • Watchers:
              0 Start watching this issue


              • Created: