[SPARK-18135] Failed to access Spark UI and HistoryServer UI - ASF JIRA

Details

Type: Bug
Status: Closed
Priority: Major
Resolution: Works for Me
Affects Version/s: 2.0.1
Fix Version/s: None
Component/s: Spark Core
Labels:
None

Target Version/s:

2.0.2, 2.1.0

Description

Environment

1. Spark package: build with Hadoop-2.7.2

./dev/make-distribution.sh --name hadoop2.7 --tgz -Psparkr -Phadoop-2.6 -Dhadoop.version=2.7.2 -Phive -Phive-thriftserver -Pyarn

2. Run Spark on Yarn(2.7.2)

Failed to access Spark UI with following exception:

16/10/27 16:00:07 WARN server.HttpChannel: /jobs/
java.lang.NoSuchMethodError: javax.servlet.http.HttpServletRequest.isAsyncStarted()Z
	at org.spark_project.jetty.servlets.gzip.GzipHandler.handle(GzipHandler.java:484)
	at org.spark_project.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
	at org.spark_project.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
	at org.spark_project.jetty.server.Server.handle(Server.java:499)
	at org.spark_project.jetty.server.HttpChannel.handle(HttpChannel.java:311)
	at org.spark_project.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
	at org.spark_project.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
	at org.spark_project.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
	at org.spark_project.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
	at java.lang.Thread.run(Thread.java:745)
16/10/27 16:00:07 WARN thread.QueuedThreadPool: 
java.lang.NoSuchMethodError: javax.servlet.http.HttpServletResponse.getStatus()I
	at org.spark_project.jetty.server.handler.ErrorHandler.handle(ErrorHandler.java:112)
	at org.spark_project.jetty.server.Response.sendError(Response.java:597)
	at org.spark_project.jetty.server.HttpChannel.handleException(HttpChannel.java:487)
	at org.spark_project.jetty.server.HttpConnection$HttpChannelOverHttp.handleException(HttpConnection.java:594)
	at org.spark_project.jetty.server.HttpChannel.handle(HttpChannel.java:387)
	at org.spark_project.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
	at org.spark_project.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
	at org.spark_project.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
	at org.spark_project.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
	at java.lang.Thread.run(Thread.java:745)

Fix: Remove servlet-api-2.5.jar from SPARK_HOME/jars

Failed to HistoryServer UI with following exceptions in log:

16/10/27 08:08:27 WARN ServletHandler: Error for /api/v1/applications
java.lang.NoSuchMethodError: javax.ws.rs.core.Application.getProperties()Ljava/util/Map;
        at org.glassfish.jersey.server.ApplicationHandler.<init>(ApplicationHandler.java:331)
        at org.glassfish.jersey.servlet.WebComponent.<init>(WebComponent.java:392)
        at org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:177)
        at org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:369)
        at javax.servlet.GenericServlet.init(GenericServlet.java:244)
        at org.spark_project.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:616)
        at org.spark_project.jetty.servlet.ServletHolder.getServlet(ServletHolder.java:472)
        at org.spark_project.jetty.servlet.ServletHolder.ensureInstance(ServletHolder.java:767)
        at org.spark_project.jetty.servlet.ServletHolder.prepare(ServletHolder.java:752)
        at org.spark_project.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
        at org.spark_project.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
        at org.spark_project.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
        at org.spark_project.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
        at org.spark_project.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.spark_project.jetty.servlets.gzip.GzipHandler.handle(GzipHandler.java:479)
        at org.spark_project.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
        at org.spark_project.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
        at org.spark_project.jetty.server.Server.handle(Server.java:499)
        at org.spark_project.jetty.server.HttpChannel.handle(HttpChannel.java:311)
        at org.spark_project.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
        at org.spark_project.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
        at org.spark_project.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
        at org.spark_project.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
        at java.lang.Thread.run(Thread.java:745)
16/10/27 08:08:33 WARN ServletHandler: /api/v1/applications
java.lang.NullPointerException
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)
        at org.spark_project.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)
        at org.spark_project.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587)
        at org.spark_project.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
        at org.spark_project.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
        at org.spark_project.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
        at org.spark_project.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.spark_project.jetty.servlets.gzip.GzipHandler.handle(GzipHandler.java:479)
        at org.spark_project.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
        at org.spark_project.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
        at org.spark_project.jetty.server.Server.handle(Server.java:499)
        at org.spark_project.jetty.server.HttpChannel.handle(HttpChannel.java:311)
        at org.spark_project.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
        at org.spark_project.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
        at org.spark_project.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
        at org.spark_project.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
        at java.lang.Thread.run(Thread.java:745)

Root cause: In my SPARK_HOME/jars, there are both jersey*-1.9.jar and jersey*-2.22.2.jar. Conflict.

Fix: remove jersey*-1.9.jar from SPARK_HOME/jars.

Confusing

I think this problem is not an accident.

Attachments

Issue Links

relates to

SPARK-15854 Spark History server gets null pointer exception

Resolved

Failed to access Spark UI and HistoryServer UI