Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-4989

wrong application configuration cause cluster down in standalone mode

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 1.0.0, 1.1.0, 1.2.0
    • 1.1.2, 1.2.1, 1.3.0
    • Deploy, Spark Core
    • None

    Description

      when enabling eventlog in standalone mode, if give the wrong configuration, the standalone cluster will down (cause master restart, lose connection with workers).

      How to reproduce: just give an invalid value to "spark.eventLog.dir", for example: spark.eventLog.dir=hdfs://tmp/logdir1, hdfs://tmp/logdir2. This will throw illegalArgumentException, which will cause the Master restart. And the whole cluster is not available.

      This is not acceptable that cluster is crashed by one application's wrong setting.

      Attachments

        Activity

          People

            liyezhang556520 Zhang, Liye
            liyezhang556520 Zhang, Liye
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: