Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-4506

Update documentation to clarify whether standalone-cluster mode is now officially supported

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.1.0, 1.1.1, 1.2.0
    • 1.1.1, 1.2.0
    • Documentation
    • None

    Description

      The "Launching Compiled Spark Applications" section of the Spark Standalone docs claims that standalone mode only supports client deploy mode:

      The spark-submit script provides the most straightforward way to submit a compiled Spark application to the cluster. For standalone clusters, Spark currently only supports deploying the driver inside the client process that is submitting the application (client deploy mode).

      It looks like standalone-cluster mode actually works (I've used it and have heard from users that are successfully using it, too).

      The current line was added in SPARK-2259 when standalone-cluster mode wasn't officially supported. It looks like SPARK-2260 fixed a number of bugs in standalone-cluster mode, so we should update the documentation if we're now ready to officially support it.

      Attachments

        Issue Links

          Activity

            People

              andrewor14 Andrew Or
              joshrosen Josh Rosen
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: