Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-3209

Preserve thread context classloader when running jobs in RemoteInterpreterServer

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.8.0
    • 0.8.0
    • zeppelin-interpreter
    • None

    Description

      Spark jobs may change current thread context classloader sometimes.

      For example in case of issue ZEPPELIN-2475 using Spark 2.2 and Scala 2.11.8 when you run DepInterpreter that will start SparkILoop --> ILoop which changes the current thread context classloader, from LauncherAppClassloader to ScalaClassloader@URLClassloader. This result in classloading problems when SparkInterpreter is trying to build up Spark Context, in case SparkInterpreter is started by scheduler on same thread.

      In short when running subsequent paragraphs, users will get an ambiguous NullPointerException, which is hard to understand as it hides the root cause of the problem. As a safety measure to prevent such cases RemoteInterpreterServer should save & restore original thread context classloader.

      Attachments

        Issue Links

          Activity

            People

              smagyari Magyari Sandor Szilard
              smagyari Magyari Sandor Szilard
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: