Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-3669

zeppelin Spark Interpreter is failing

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.8.0
    • None
    • Interpreters

    Description

      Hi,

      While running first spark program in zeppelin, it is failing to create spark interpreter process. Below are the error details. please let me know how to fix this issue.

       

      INFO [2018-07-29 00:06:05,323] ({pool-2-thread-2} Paragraph.java[jobRun]:380) - Run paragraph [paragraph_id: 20180728-222626_1797203208, interpreter: spark, note_id: 2DN1RQXVR, user: anonymous]
      INFO [2018-07-29 00:06:05,323] ({pool-2-thread-2} ManagedInterpreterGroup.java[getOrCreateInterpreterProcess]:61) - Create InterpreterProcess for InterpreterGroup: spark:shared_process
      INFO [2018-07-29 00:06:05,325] ({pool-2-thread-2} ShellScriptLauncher.java[launch]:48) - Launching Interpreter: spark
      INFO [2018-07-29 00:06:05,351] ({pool-2-thread-2} SparkInterpreterLauncher.java[buildEnvFromProperties]:103) - Run Spark under non-secure mode as no keytab and principal is specified
      DEBUG [2018-07-29 00:06:05,352] ({pool-2-thread-2} SparkInterpreterLauncher.java[buildEnvFromProperties]:105) - buildEnvFromProperties: {ZEPPELIN_SPARK_CONF= --master local[*] --conf spark.yarn.dist.archives=C:\spark\R\lib\sparkr.zip#sparkr --conf spark.app.name='Zeppelin', SPARK_HOME=C:\spark}
      INFO [2018-07-29 00:06:05,361] ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:115) - Thrift server for callback will start. Port: 57136
      DEBUG [2018-07-29 00:06:05,371] ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:153) - callbackServer is serving now
      INFO [2018-07-29 00:06:05,380] ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:190) - Run interpreter process [C:\zeppelin\zeppelin-0.8.0-bin-all\zeppelin-0.8.0-bin-all\bin\interpreter.cmd, -d, C:\zeppelin\zeppelin-0.8.0-bin-all\zeppelin-0.8.0-bin-all/interpreter/spark, -c, 10.120.44.23, -p, 57136, -r, :, -l, C:\zeppelin\zeppelin-0.8.0-bin-all\zeppelin-0.8.0-bin-all/local-repo/spark, -g, spark]
      DEBUG [2018-07-29 00:06:09,625] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - Warning: Local jar C:\Users\vv\57136 does not exist, skipping.
      DEBUG [2018-07-29 00:06:09,626] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - Warning: Local jar C:\Users\vv\10.120.44.23 does not exist, skipping.
      DEBUG [2018-07-29 00:06:09,627] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - java.lang.ClassNotFoundException: org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer
      DEBUG [2018-07-29 00:06:09,628] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
      DEBUG [2018-07-29 00:06:09,630] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
      DEBUG [2018-07-29 00:06:09,632] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      DEBUG [2018-07-29 00:06:09,633] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at java.lang.Class.forName0(Native Method)
      DEBUG [2018-07-29 00:06:09,634] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at java.lang.Class.forName(Class.java:348)
      DEBUG [2018-07-29 00:06:09,635] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at org.apache.spark.util.Utils$.classForName(Utils.scala:238)
      DEBUG [2018-07-29 00:06:09,641] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:851)
      DEBUG [2018-07-29 00:06:09,642] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)
      DEBUG [2018-07-29 00:06:09,644] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)
      DEBUG [2018-07-29 00:06:09,645] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
      DEBUG [2018-07-29 00:06:09,647] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
      DEBUG [2018-07-29 00:06:09,656] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - 2018-07-29 00:06:09 INFO ShutdownHookManager:54 - Shutdown hook called
      DEBUG [2018-07-29 00:06:09,659] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - 2018-07-29 00:06:09 INFO ShutdownHookManager:54 - Deleting directory C:\Users\vv\AppData\Local\Temp\spark-427c3202-c243-4761-86ce-ea51a27a881c
      INFO [2018-07-29 00:06:09,747] ({Exec Default Executor} RemoteInterpreterManagedProcess.java[onProcessFailed]:250) - Interpreter process failed {}
      org.apache.commons.exec.ExecuteException: Process exited with an error: 101 (Exit value: 101)
      at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)
      at org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48)
      at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200)
      at java.lang.Thread.run(Thread.java:748)
      ERROR [2018-07-29 00:07:05,382] ({pool-2-thread-2} Job.java[run]:190) - Job failed
      java.lang.RuntimeException: Warning: Local jar C:\Users\vv\57136 does not exist, skipping.
      Warning: Local jar C:\Users\vv\10.120.44.23 does not exist, skipping.
      java.lang.ClassNotFoundException: org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer
      at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      at java.lang.Class.forName0(Native Method)
      at java.lang.Class.forName(Class.java:348)
      at org.apache.spark.util.Utils$.classForName(Utils.scala:238)
      at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:851)
      at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)
      at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)
      at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
      at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
      2018-07-29 00:06:09 INFO ShutdownHookManager:54 - Shutdown hook called

       

      Thanks

      VV

      Attachments

        Activity

          People

            Unassigned Unassigned
            viswanatharv VV
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: