Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-6058

Log the error for the EXIT_EXCEPTION_USER_CLASS exit code

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 1.2.1
    • 1.3.0
    • YARN
    • None

    Description

      When running in the yarn-cluster mode, the user cannot find the stack trace of the driver error in the log.

      In https://github.com/apache/spark/blob/8cd1692c9092150107bed27777951633cbf945f6/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala#L488 , it will rethrow `cause`. However, because ApplicationMaster doesn't set SparkUncaughtExceptionHandler. The error won't be logged.

      To view this error in the log, we should call `logError` to log it.

      Attachments

        Issue Links

          Activity

            People

              zsxwing Shixiong Zhu
              zsxwing Shixiong Zhu
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: