Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-5825

Zeppelin's spark interpreter erroring out when running with Standalone Spark cluster in cluster deploy mode

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.10.1
    • None
    • spark
    • None

    Description

      Zeppelin Run Mode: local

      Trying to run the zeppelin in cluster deploy mode of spark on a spark standalone cluster. 

      Error in Spark Driver : 

       WARN [2022-09-22 08:01:38,996] ({rpc-client-5-1} TransportChannelHandler.java[exceptionCaught]:87) - Exception in connection from spark-master/100.89.25.74:6066
      java.lang.IllegalArgumentException: Too large frame: 5211883372140375593
      	at org.sparkproject.guava.base.Preconditions.checkArgument(Preconditions.java:119)
      	at org.apache.spark.network.util.TransportFrameDecoder.decodeNext(TransportFrameDecoder.java:148)
      	at org.apache.spark.network.util.TransportFrameDecoder.channelRead(TransportFrameDecoder.java:98)
      	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
      	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
      	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
      	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
      	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
      	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
      	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
      	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
      	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719)
      	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
      	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
      	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
      	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
      	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
      	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
      	at java.lang.Thread.run(Thread.java:750)
      ERROR [2022-09-22 08:01:38,998] ({rpc-client-5-1} TransportResponseHandler.java[exceptionCaught]:155) - Still have 1 requests outstanding when connection from spark-master/100.89.25.74:6066 is closed
       WARN [2022-09-22 08:01:38,999] ({dispatcher-event-loop-0} Logging.scala[logWarning]:69) - Could not connect to spark-master:6066: java.lang.IllegalArgumentException: Too large frame: 5211883372140375593
       WARN [2022-09-22 08:01:38,999] ({appclient-register-master-threadpool-0} Logging.scala[logWarning]:90) - Failed to connect to master spark-master:6066
      org.apache.spark.SparkException: Exception thrown in awaitResult: 
      	at org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:301)
      	at org.apache.spark.rpc.RpcTimeout.awaitResult(RpcTimeout.scala:75)
      	at org.apache.spark.rpc.RpcEnv.setupEndpointRefByURI(RpcEnv.scala:101)
      	at org.apache.spark.rpc.RpcEnv.setupEndpointRef(RpcEnv.scala:109)
      	at org.apache.spark.deploy.client.StandaloneAppClient$ClientEndpoint$$anon$1.run(StandaloneAppClient.scala:107)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:750)
      Caused by: java.lang.IllegalArgumentException: Too large frame: 5211883372140375593
      	at org.sparkproject.guava.base.Preconditions.checkArgument(Preconditions.java:119)
      	at org.apache.spark.network.util.TransportFrameDecoder.decodeNext(TransportFrameDecoder.java:148)
      	at org.apache.spark.network.util.TransportFrameDecoder.channelRead(TransportFrameDecoder.java:98)
      	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
      	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
      	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
      	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
      	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
      	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
      	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
      	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
      	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719)
      	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
      	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
      	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
      	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
      	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
      	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
      	... 1 more
       INFO [2022-09-22 08:01:58,990] ({appclient-register-master-threadpool-0} Logging.scala[logInfo]:57) - Connecting to master spark://spark-master:6066...
       INFO [2022-09-22 08:01:58,993] ({netty-rpc-connection-1} TransportClientFactory.java[createClient]:206) - Found inactive connection to spark-master/100.89.25.74:6066, creating a new one.
       INFO [2022-09-22 08:01:59,000] ({netty-rpc-connection-1} TransportClientFactory.java[createClient]:310) - Successfully created connection to spark-master/100.89.25.74:6066 after 6 ms (0 ms spent in bootstraps)

      Attachments

        1. local-cluster-spark-zeppelin.zip
          9 kB
          Ankit Prakash Gupta
        2. spark-interpreter.json
          14 kB
          Ankit Prakash Gupta

        Activity

          People

            Unassigned Unassigned
            infoankitp Ankit Prakash Gupta
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: