Uploaded image for project: 'Marathon'
  1. Marathon
  2. MARATHON-4072

HTTP 500 error when app.ports has fewer elements than Health Check needs

    Details

    • Type: Task
    • Status: Resolved
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Component/s: None

      Description

      On app creation, if a health check references a portIndex outside the bounds of ports, a HTTP 500 is thrown. This is true even if there are sufficient portMappings to satisfy the index reference.

      Here's the app:

       json
      {
        "id": "bridged-webapp",
        "cmd": "python3 -m http.server 8080",
        "cpus": 0.5,
        "mem": 64.0,
        "instances": 2,
        "container": {
          "type": "DOCKER",
          "docker": {
            "image": "python:3",
            "network": "BRIDGE",
            "portMappings": [
              { "containerPort": 8080, "hostPort": 0, "protocol": "tcp"},
              { "containerPort": 161, "hostPort": 0, "protocol": "udp"}
            ]
          }
        },
        "healthChecks": [
          {
            "protocol": "HTTP",
            "portIndex": 0,
            "path": "/",
            "gracePeriodSeconds": 5,
            "intervalSeconds": 20,
            "maxConsecutiveFailures": 3
          }
        ],
        "ports": []
      }
      

      Here's the response:

       console
      HTTP/1.1 500 Server Error
      Cache-Control: must-revalidate,no-cache,no-store
      Content-Length: 1280
      Content-Type: text/html;charset=ISO-8859-1
      Server: Jetty(8.y.z-SNAPSHOT)
      
      <html>
      <head>
      <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
      <title>Error 500 Server Error</title>
      </head>
      <body>
      <h2>HTTP ERROR: 500</h2>
      <p>Problem accessing /v2/apps. Reason:
      <pre>    Server Error</pre></p>
      <hr /><i><small>Powered by Jetty://</small></i>
      </body>
      </html>
      
      

      Here's the logged exception:

       console
      2014-09-29_22:39:38.83939 [2014-09-29 22:39:38,836] WARN Error for /v2/apps (org.eclipse.jetty.servlet.ServletHandler:542)
      2014-09-29_22:39:38.83942 java.lang.AssertionError: assertion failed: Port indices must address an element of this app's ports array.
      2014-09-29_22:39:38.83943   at scala.Predef$.assert(Predef.scala:179)
      2014-09-29_22:39:38.83944   at mesosphere.marathon.state.AppDefinition.<init>(AppDefinition.scala:73)
      2014-09-29_22:39:38.83945   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      2014-09-29_22:39:38.83946   at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
      2014-09-29_22:39:38.83949   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      2014-09-29_22:39:38.83950   at java.lang.reflect.Constructor.newInstance(Constructor.java:534)
      2014-09-29_22:39:38.83951   at scala.reflect.runtime.JavaMirrors$JavaMirror$JavaConstructorMirror.apply(JavaMirrors.scala:444)
      2014-09-29_22:39:38.83952   at mesosphere.reflect.CaseClassFactory.buildWith(CaseClassFactory.scala:38)
      2014-09-29_22:39:38.83953   at mesosphere.jackson.CaseClassValueInstantiator.createFromObjectWith(CaseClassValueInstantiator.scala:116)
      2014-09-29_22:39:38.83955   at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:167)
      2014-09-29_22:39:38.83957   at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:353)
      2014-09-29_22:39:38.83958   at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1058)
      2014-09-29_22:39:38.83959   at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:268)
      2014-09-29_22:39:38.83960   at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:124)
      2014-09-29_22:39:38.83961   at com.fasterxml.jackson.databind.ObjectReader._bind(ObjectReader.java:1232)
      2014-09-29_22:39:38.83962   at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:676)
      2014-09-29_22:39:38.83962   at com.fasterxml.jackson.jaxrs.base.ProviderBase.readFrom(ProviderBase.java:764)
      2014-09-29_22:39:38.83963   at mesosphere.chaos.validation.JacksonMessageBodyProvider.readFrom(JacksonMessageBodyProvider.java:59)
      2014-09-29_22:39:38.83964   at com.sun.jersey.spi.container.ContainerRequest.getEntity(ContainerRequest.java:490)
      2014-09-29_22:39:38.83965   at com.sun.jersey.server.impl.model.method.dispatch.EntityParamDispatchProvider$EntityInjectable.getValue(EntityParamDispatchProvider.java:123)
      2014-09-29_22:39:38.83967   at com.sun.jersey.server.impl.inject.InjectableValuesProvider.getInjectableValues(InjectableValuesProvider.java:86)
      2014-09-29_22:39:38.83968   at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$EntityParamInInvoker.getParams(AbstractResourceMethodDispatchProvider.java:153)
      2014-09-29_22:39:38.83969   at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:203)
      2014-09-29_22:39:38.83970   at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
      2014-09-29_22:39:38.83971   at com.codahale.metrics.jersey.InstrumentedResourceMethodDispatchProvider$TimedRequestDispatcher.dispatch(InstrumentedResourceMethodDispatchProvider.java:30)
      2014-09-29_22:39:38.83971   at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
      2014-09-29_22:39:38.83972   at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
      2014-09-29_22:39:38.83973   at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
      2014-09-29_22:39:38.83975   at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
      2014-09-29_22:39:38.83976   at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
      2014-09-29_22:39:38.83977   at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
      2014-09-29_22:39:38.83978   at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
      2014-09-29_22:39:38.83979   at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
      2014-09-29_22:39:38.83979   at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
      2014-09-29_22:39:38.83980   at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:540)
      2014-09-29_22:39:38.83981   at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:715)
      2014-09-29_22:39:38.83982   at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
      2014-09-29_22:39:38.83983   at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
      2014-09-29_22:39:38.83984   at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
      2014-09-29_22:39:38.83986   at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
      2014-09-29_22:39:38.83987   at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
      2014-09-29_22:39:38.83988   at mesosphere.marathon.api.LeaderProxyFilter.doFilter(LeaderProxyFilter.scala:56)
      2014-09-29_22:39:38.83989   at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
      2014-09-29_22:39:38.83990   at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
      2014-09-29_22:39:38.83991   at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
      2014-09-29_22:39:38.83991   at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
      2014-09-29_22:39:38.83992   at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
      2014-09-29_22:39:38.83993   at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
      2014-09-29_22:39:38.83994   at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
      2014-09-29_22:39:38.83995   at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
      2014-09-29_22:39:38.83996   at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
      2014-09-29_22:39:38.83998   at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
      2014-09-29_22:39:38.83998   at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
      2014-09-29_22:39:38.83999   at com.codahale.metrics.jetty8.InstrumentedHandler.handle(InstrumentedHandler.java:187)
      2014-09-29_22:39:38.84000   at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
      2014-09-29_22:39:38.84001   at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
      2014-09-29_22:39:38.84002   at org.eclipse.jetty.server.Server.handle(Server.java:370)
      2014-09-29_22:39:38.84003   at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
      2014-09-29_22:39:38.84003   at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:960)
      2014-09-29_22:39:38.84004   at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1021)
      2014-09-29_22:39:38.84005   at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
      2014-09-29_22:39:38.84006   at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
      2014-09-29_22:39:38.84007   at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
      2014-09-29_22:39:38.84009   at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:668)
      2014-09-29_22:39:38.84010   at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
      2014-09-29_22:39:38.84011   at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
      2014-09-29_22:39:38.84012   at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
      2014-09-29_22:39:38.84012   at java.lang.Thread.run(Thread.java:701)
      

      [~ConnorDoyle] mentioned the root cause is likely the JSON validator.

      Additionally, it would be helpful to note in the docs that ports is ignored if portMappings is provided.

        Attachments

          Activity

            People

            • Assignee:
              GitHub_ConnorDoyle Connor Doyle (Inactive)
              Reporter:
              GitHub_mbabineau Mike Babineau (Inactive)
              Team:
              Orchestration Team
              Watchers:
            • Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: