Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
3.4.0, 3.4.1, 3.5.0
-
None
Description
When a job is submitted in standalone mode using spark launch, sparkLancher returns a successful execution and then a failed execution. Examples are as follows
log: Spark App Id [app-20230922160022-0006] State Changed. State [FINISHED]
23/09/22 16:01:09 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
23/09/22 16:01:10 INFO MemoryStore: MemoryStore cleared
23/09/22 16:01:10 INFO BlockManager: BlockManager stopped
23/09/22 16:01:10 INFO BlockManagerMaster: BlockManagerMaster stopped
23/09/22 16:01:10 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
23/09/22 16:01:10 INFO SparkContext: Successfully stopped SparkContext
23/09/22 16:01:10 INFO ShutdownHookManager: Shutdown hook called
23/09/22 16:01:10 INFO ShutdownHookManager: Deleting directory /tmp/spark-e015933f-a220-45a8-9d73-650b8bd8a337
23/09/22 16:01:10 INFO ShutdownHookManager: Deleting directory /mnt/tmp/spark-8879f1d8-1f21-4b52-bca1-d3c66af6f754
23/09/22 16:01:11 INFO log: Spark App Id [app-20230922160022-0006] State Changed. State [FAILED]