Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-13740

Corrupt 'spark-log4j-properties' for Spark service

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 2.1.0
    • None
    • ambari-admin, ambari-client
    • None
    • Hortonworks sandbox 2.3.1; HDP-2.3.0.0-2557

    Description

      It seems like the 'spark-log4j-properties' configuration passed from Ambari | Services | Spark | Configs is getting corrupt and reformatted into one line without LF. The log4j config file is getting compressed into a field, with all LF characters dropped out. This configuration apparently cannot be changed into normal/standard multi line configuration that would be meaningful to Spark.
      This way it seems like the whole log4j configuration is being ignored completely. If not due to the syntactical sanity, then at least due to the first character is '#', which applies to the whole line. The default Spark loglevel is set to INFO, so consequently console prints whole number of INFO messages for each statement passed to the console. Consequently further, the loglevel cannot be changed to WARN or ERROR and INFO lines cannot be suppressed from the console.

      --> The log4j.properties.template configuration looks like this:

      1. Set everything to be logged to the console
        log4j.rootCategory=INFO, console
        log4j.appender.console=org.apache.log4j.ConsoleAppender
        log4j.appender.console.target=System.err
        log4j.appender.console.layout=org.apache.log4j.PatternLayout
        log4j.appender.console.layout.ConversionPattern=%d {yy/MM/dd HH:mm:ss}

        %p %c

        {1}: %m%n

        # Settings to quiet third party logs that are too verbose
        log4j.logger.org.eclipse.jetty=WARN
        log4j.logger.org.eclipse.jetty.util.component.AbstractLifeCycle=ERROR
        log4j.logger.org.apache.spark.repl.SparkIMain$exprTyper=INFO
        log4j.logger.org.apache.spark.repl.SparkILoop$SparkILoopInterpreter=INFO

        --> The log4j.properties configuration looks like this:

        # Set everything to be logged to the consolelog4j.rootCategory=WARN, consolelog4j.appender.console=org.apache.log4j.ConsoleAppenderlog4j.appender.console.target=System.errlog4j.appender.console.layout=org.apache.log4j.PatternLayoutlog4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}

        : %m%n# Settings to quiet third party logs that are too verboselog4j.logger.org.eclipse.jetty=WARNlog4j.logger.org.eclipse.jetty.util.component.AbstractLifeCycle=ERRORlog4j.logger.org.apache.spark.repl.SparkIMain$exprTyper=INFOlog4j.logger.org.apache.spark.repl.SparkILoop$SparkILoopInterpreter=INFO

      --> This issue leads to the ### INFO messages that log4j prints in Spark console, without possibility to change the loglevel:

      [root@sandbox ~]# spark-shell --master yarn-client --driver-memory 512m --executor-memory 512m
      log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
      log4j:WARN Please initialize the log4j system properly.
      log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
      Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
      15/11/05 12:34:41 INFO SecurityManager: Changing view acls to: root
      15/11/05 12:34:41 INFO SecurityManager: Changing modify acls to: root
      15/11/05 12:34:41 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); users with modify permissions: Set(root)
      15/11/05 12:34:41 INFO HttpServer: Starting HTTP Server
      15/11/05 12:34:41 INFO Server: jetty-8.y.z-SNAPSHOT
      15/11/05 12:34:41 INFO AbstractConnector: Started SocketConnector@0.0.0.0:54564
      15/11/05 12:34:41 INFO Utils: Successfully started service 'HTTP class server' on port 54564.
      Welcome to
      ____ __
      / _/_ ___ ____/ /_
      \ \/ _ \/ _ `/ __/ '/
      /__/ ./_,// //_\ version 1.3.1
      /_/

      Using Scala version 2.10.4 (OpenJDK 64-Bit Server VM, Java 1.7.0_85)
      Type in expressions to have them evaluated.
      Type :help for more information.
      15/11/05 12:34:44 INFO SparkContext: Running Spark version 1.3.1
      15/11/05 12:34:44 INFO SecurityManager: Changing view acls to: root
      15/11/05 12:34:44 INFO SecurityManager: Changing modify acls to: root
      15/11/05 12:34:44 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); users with modify permissions: Set(root)
      15/11/05 12:34:44 INFO Slf4jLogger: Slf4jLogger started
      15/11/05 12:34:44 INFO Remoting: Starting remoting
      15/11/05 12:34:44 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriver@sandbox.hortonworks.com:35020]
      15/11/05 12:34:44 INFO Utils: Successfully started service 'sparkDriver' on port 35020.
      15/11/05 12:34:44 INFO SparkEnv: Registering MapOutputTracker
      15/11/05 12:34:44 INFO SparkEnv: Registering BlockManagerMaster
      15/11/05 12:34:44 INFO DiskBlockManager: Created local directory at /tmp/spark-3b14e9fc-73b1-45e7-96f3-f0ee48f3a238/blockmgr-ffd0619a-5fbc-4ccc-bcef-ce8560e03cd7
      15/11/05 12:34:44 INFO MemoryStore: MemoryStore started with capacity 265.4 MB
      15/11/05 12:34:45 INFO HttpFileServer: HTTP File server directory is /tmp/spark-7648f575-a8e3-4b25-945f-b4ded8631592/httpd-ec5526bb-4de6-42d6-83aa-a1ba0a2ca5dd
      15/11/05 12:34:45 INFO HttpServer: Starting HTTP Server
      15/11/05 12:34:45 INFO Server: jetty-8.y.z-SNAPSHOT
      15/11/05 12:34:45 INFO AbstractConnector: Started SocketConnector@0.0.0.0:58853
      15/11/05 12:34:45 INFO Utils: Successfully started service 'HTTP file server' on port 58853.
      15/11/05 12:34:45 INFO SparkEnv: Registering OutputCommitCoordinator
      1
      ...
      ...

      Attachments

        Activity

          People

            Unassigned Unassigned
            d6z7 Drazen Zubovic
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: