Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-16501

spark.mesos.secret exposed on UI and command line

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.6.2
    • 2.4.0
    • Spark Submit, Web UI

    Description

      There are two related problems with spark.mesos.secret:

      1) The web UI shows its value in the "environment" tab
      2) Passing it as a command-line option to spark-submit (or creating a SparkContext from python, with the effect of launching spark-submit) exposes it to "ps"

      I'll be happy to submit a patch but I could use some advice first.

      The first problem is easy enough, just don't show that value in the UI

      For the second problem, I'm not sure what the best solution is. A "spark.mesos.secret-file" parameter would let the user store the secret in a non-world-readable file. Alternatively, the mesos secret could be obtained from the environment, which other users don't have access to. Either solution would work in client mode, but I don't know if they're workable in cluster mode.

      Attachments

        Issue Links

          Activity

            People

              rvesse Rob Vesse
              edaniel Eric Daniel
              Votes:
              2 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: