Uploaded image for project: 'DC/OS'
  1. DC/OS
  2. DCOS_OSS-3441

Deploying Spark dispatcher under Enterprise Edition in strict mode fails to start

    Details

      Description

      I followed the instructions for configuring Spark for running in strict mode in Enterprise found here: https://docs.mesosphere.com/service-docs/spark/spark-auth/

       

      I only did the strict instructions.  In the end I wound up with a package configuration that looked like:

      {
        "service": {
          "principal": "spark-principal",
          "secret_name": "spark/private-key",
          "user": "nobody",
          "log-level": "WARN"
        }
      }
      

      When I deploy with:

      dcos package install --options=spark.json spark
      

      The dispatcher does not start and errors out with the error in the attached screenshot, but the important part is:

      + grep -v '#https#' /etc/nginx/conf.d/spark.conf.template
      + sed s,#http#,,
      /sbin/init.sh: line 24: /etc/nginx/conf.d/spark.conf: Permission denied
      

      This error seems to be directly the result of: https://github.com/mesosphere/spark-build/blob/master/docker/Dockerfile#L75-L81

      # Commenting these for now, because we're running Spark as root in strict mode
      # RUN chmod -R ugo+rw /etc/nginx
      # RUN chmod -R ugo+rw /etc/service
      # RUN chmod -R ugo+rw /var/lib/
      # RUN chmod -R ugo+rw /var/run/
      # RUN chmod -R ugo+rw /var/log/
      # RUN chmod -R ugo+rw /opt/spark/dist
      

      This comment is directly the opposite of what appears in the documentation:

      Note: At this time, Spark tasks other than the dispatcher must run under the root user.
      

      and the example configuration given is:

      {
        "service": {
          "principal": "spark-principal",
          "secret_name": "spark/<secret-name>",
          "user": "nobody"
        }
      }
      

      I have tried to deploy the dispatcher under `root` but the deployment does not succeed.  I presume I need some special ACL magic that I don't posses.

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              jimpowers Jim Powers
              Team:
              Data Services Team
              Watchers:
              Arthur Rand, Chris Lambert (Inactive), Jim Powers, Mahendra Kutare (Inactive), skonto, Stavros Kontopoulos, Susan X. Huynh (Inactive)
            • Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: