Description
1. create a spark job workflow in Oozie
2. add --conf spark.executor.extraClassPath=/XXX/XXX in job confuration
3. submit the job
Results:
the Oozie workflow failed and the error log is as follows. it has an addtion error "--conf".
--conf
--conf
spark.executor.extraClassPath=aaa:$PWD/*
--conf
spark.driver.extraClassPath=$PWD/*
Error: Spark config without '=': --conf
Run with --help for usage help or --verbose for debug output
Intercepting System.exit(1)
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain], exit code [1]
Attachments
Issue Links
- duplicates
-
OOZIE-2984 Parse spark-defaults.conf values with spaces without needing the quotes
- Closed