Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-4632

No such element exception:spark.driver.cores

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • v4.0.0-alpha
    • v4.0.0-alpha
    • Spark Engine
    • None
    • Sprint 53

    Description

      When submit a build job, it throws an exception. But the build job still can run successfully.
      20/07/10 14:06:29 WARN SparkApplication: Error occurred when check resource. Ignore it and try to submit this job.
      java.util.NoSuchElementException: spark.driver.cores
      at org.apache.spark.SparkConf$$anonfun$get$1.apply(SparkConf.scala:245)
      at org.apache.spark.SparkConf$$anonfun$get$1.apply(SparkConf.scala:245)
      at scala.Option.getOrElse(Option.scala:121)
      at org.apache.spark.SparkConf.get(SparkConf.scala:245)
      at org.apache.spark.utils.ResourceUtils$.checkResource(ResourceUtils.scala:71)
      at org.apache.spark.utils.ResourceUtils.checkResource(ResourceUtils.scala)
      at org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:156)
      at org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:76)
      at org.apache.spark.application.JobWorker$$anon$2.run(JobWorker.scala:55)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)

      Attachments

        Activity

          People

            wangrupeng Wang Rupeng
            wangrupeng Wang Rupeng
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: