Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-20369

Need hdfs-site for saving ranger audits to hdfs in namenode HA env

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.5.0
    • 2.5.0
    • ambari-server

    Description

      For KNOX and RANGER_KMS services which supports ranger plugin, need to have hdfs-site.xml available in respective services conf directory for saving ranger audits to hdfs in namenode HA env.

      Below error logs are found, if hdfs-site.xml is not available,

      2017-03-01 18:48:50,150 ERROR provider.BaseAuditHandler (BaseAuditHandler.java:logError(327)) - Error writing to log file.
      java.lang.IllegalArgumentException: java.net.UnknownHostException: mycluster
      	at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:438)
      	at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:311)
      	at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
      	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:690)
      	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:631)
      	at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:160)
      	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2795)
      	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:99)
      	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2829)
      	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2811)
      	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:390)
      	at org.apache.ranger.audit.destination.HDFSAuditDestination.getLogFileStream(HDFSAuditDestination.java:271)
      	at org.apache.ranger.audit.destination.HDFSAuditDestination.access$000(HDFSAuditDestination.java:43)
      	at org.apache.ranger.audit.destination.HDFSAuditDestination$1.run(HDFSAuditDestination.java:157)
      	at org.apache.ranger.audit.destination.HDFSAuditDestination$1.run(HDFSAuditDestination.java:154)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:422)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
      	at org.apache.ranger.audit.provider.MiscUtil.executePrivilegedAction(MiscUtil.java:523)
      	at org.apache.ranger.audit.destination.HDFSAuditDestination.logJSON(HDFSAuditDestination.java:154)
      	at org.apache.ranger.audit.queue.AuditFileSpool.sendEvent(AuditFileSpool.java:880)
      	at org.apache.ranger.audit.queue.AuditFileSpool.runLogAudit(AuditFileSpool.java:828)
      	at org.apache.ranger.audit.queue.AuditFileSpool.run(AuditFileSpool.java:758)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: java.net.UnknownHostException: mycluster
      	... 24 more
      2017-03-01 18:48:50,151 ERROR queue.AuditFileSpool (AuditFileSpool.java:logError(710)) - Error sending logs to consumer. provider=knox.async.multi_dest.batch, consumer=knox.async.multi_dest.batch.hdfs{

      Attachments

        1. AMBARI-20369.patch
          6 kB
          Mugdha Varadkar

        Issue Links

          Activity

            People

              mugdha.varadkar Mugdha Varadkar
              mugdha.varadkar Mugdha Varadkar
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: