Description
When running in the yarn-cluster mode, the user cannot find the stack trace of the driver error in the log.
In https://github.com/apache/spark/blob/8cd1692c9092150107bed27777951633cbf945f6/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala#L488 , it will rethrow `cause`. However, because ApplicationMaster doesn't set SparkUncaughtExceptionHandler. The error won't be logged.
To view this error in the log, we should call `logError` to log it.
Attachments
Issue Links
- relates to
-
SPARK-6018 NoSuchMethodError in Spark app is swallowed by YARN AM
- Closed
- links to