Uploaded image for project: 'Marathon'
  1. Marathon
  2. MARATHON-8428

dcos marathon pod list throws Internal Server Error

    Details

    • Sprint:
      Marathon 2018-29
    • Story Points:
      1
    • Build artifact:
      Marathon-v1.7.174

      Description

      Just upgraded from 1.11.1 to 1.11.3.

      Tried to demonstrate the prometheus integration to a user as per the documentation. The DC/OS UI hung up and I am not able to list or kill the pod that is causing this behavior.

      Running 

      dcos marathon pod list 

      Throws

      Error: task startedAt expected to not me empty

      Here is the stack trace of the CLI call:

      dcos --log-level="debug" marathon pod list
      MainThread: 2018-06-27 16:46:11,605 dcos/marathon.py:create_client:27 - Creating marathon client with: 'http://mesosmaster1.lvs.nimblestorage.com/service/marathon/'
      MainThread: 2018-06-27 16:46:11,606 dcos/http.py:_request:163 - Sending HTTP ['head'] to ['http://mesosmaster1.lvs.nimblestorage.com/service/marathon/v2/pods']: {'Accept': 'application/json'}
      MainThread: 2018-06-27 16:46:11,608 cli/env/lib/python3.5/site-packages/urllib3/connectionpool.py:_new_conn:208 - Starting new HTTP connection (1): mesosmaster1.lvs.nimblestorage.com
      MainThread: 2018-06-27 16:46:11,647 cli/env/lib/python3.5/site-packages/urllib3/connectionpool.py:_make_request:396 - http://mesosmaster1.lvs.nimblestorage.com:80 "HEAD /service/marathon/v2/pods HTTP/1.1" 200 0
      MainThread: 2018-06-27 16:46:11,648 dcos/http.py:_request:193 - Received HTTP response [200]: {'X-Marathon-Leader': 'http://10.175.1.240:8080', 'Connection': 'keep-alive', 'Expires': '0', 'Content-Length': '0', 'Date': 'Wed, 27 Jun 2018 23:46:11 GMT', 'Cache-Control': 'no-cache, no-store, must-revalidate', 'Content-Type': 'application/json', 'Pragma': 'no-cache', 'Server': 'openresty'}
      MainThread: 2018-06-27 16:46:11,648 dcos/util.py:timer:609 - duration: dcos.http._request: 0.04s
      MainThread: 2018-06-27 16:46:11,650 dcos/http.py:_request:163 - Sending HTTP ['get'] to ['http://mesosmaster1.lvs.nimblestorage.com/service/marathon/v2/pods/::status']: {'Accept': 'application/json'}
      MainThread: 2018-06-27 16:46:11,651 cli/env/lib/python3.5/site-packages/urllib3/connectionpool.py:_new_conn:208 - Starting new HTTP connection (1): mesosmaster1.lvs.nimblestorage.com
      MainThread: 2018-06-27 16:46:11,694 cli/env/lib/python3.5/site-packages/urllib3/connectionpool.py:_make_request:396 - http://mesosmaster1.lvs.nimblestorage.com:80 "GET /service/marathon/v2/pods/::status HTTP/1.1" 500 53
      MainThread: 2018-06-27 16:46:11,695 dcos/http.py:_request:193 - Received HTTP response [500]: {'X-Marathon-Leader': 'http://10.175.1.240:8080', 'Connection': 'keep-alive', 'Expires': '0', 'Content-Length': '53', 'Date': 'Wed, 27 Jun 2018 23:46:11 GMT', 'Cache-Control': 'no-cache, no-store, must-revalidate', 'Content-Type': 'application/json', 'Pragma': 'no-cache', 'Server': 'openresty'}
      MainThread: 2018-06-27 16:46:11,695 dcos/util.py:timer:609 - duration: dcos.http._request: 0.05s
      MainThread: 2018-06-27 16:46:11,696 dcos/rpcclient.py:http_req:131 - DCOS Error: Internal Server Error
      {"message":"task startedAt expected to not me empty"}
      Error: task startedAt expected to not me empty

       

        Attachments

          Activity

            People

            • Assignee:
              matthias.eichstedt Matthias Eichstedt
              Reporter:
              jneytchev Julian Neytchev (Inactive)
              Team:
              Orchestration Team
              Watchers:
              Brian Lewis, Karsten Jeschkies, Matthias Eichstedt, Mergebot
            • Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: