Uploaded image for project: 'DC/OS'
  1. DC/OS
  2. DCOS_OSS-4645

Jobs menu failing because of bad request to Metronome

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Medium
    • Resolution: Done
    • Affects Version/s: DC/OS 1.12.0
    • Fix Version/s: DC/OS 1.12.1
    • Component/s: dcos-ui
    • Labels:
      None

      Description

      Hello

       

      When we upgraded to 1.12.0 we noticed a problem with the DC/OS UI: The Jobs panel was giving an error (as shown in the screenshot below).

       

       

      The cause of this was a HTTP 400 response for a GET request to

      https://MASTER_URL/service/metronome/v1/jobs?embed=activeRuns&embed=schedules&embed=historySummary

       with the response being

      {"message":"Invalid Json: No content to map due to end-of-input\n at [Source: (akka.util.ByteIterator$ByteArrayIterator$$anon$1); line: 1, column: 0]","requestPath":"/v1/jobs"}.

       

      After some investigation we noticed that a "Content-Type: application/json; charset=utf-8" header was being sent with this GET request, which is puzzling since the request contains no request body, meaning the JSON decoder was failing in Metronome. If the Content-Type header is removed, then the request works correctly.

       

      As a disgusting hotfix we have patched the adminrouter lua library to scrub the header out of the request to metronome, but that's mostly because I do not know enough javascript to fix it in the dcos-ui package. The patch (for people equally uncomfortable in javascript) is below.

       

      diff original_service.lua /opt/mesosphere/active/adminrouter/nginx/conf/lib/service.lua
      362a363,365
      >       if service_name == 'metronome' and ngx.req.get_method() == 'GET' then
      >           ngx.req.clear_header("Content-Type")
      >       end

      (for clarity, this is in the "recursive_resolve function, just near the end)

       

      It should be worth mentioning that we first noticed this issue when we upgraded, which was shortly after 1.12.0's release. However it disappeared for some time, and I'm not sure why. It reappeared today which meant I went to investigate.

       

        Attachments

          Activity

            People

            • Assignee:
              drozhkov Dima Rozhkov
              Reporter:
              douglastemple Douglas Temple
              Team:
              Frontend Team
              Watchers:
              Douglas Temple, GeorgiSTodorov, Julian Gieseke
            • Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: