Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5694

MR AM container syslog is empty

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.2.0
    • 2.3.0
    • None
    • None
    • Reviewed

    Description

      When the property "mapreduce.task.userlog.limit.kb" is set non-zero in mapped-site.xml, AM container syslog remains empty. Without the log, it is hard to identify the cause of any MR AM failure.
      However, if "mapreduce.task.userlog.limit.kb" is not set (or defaulted to 0), syslog contents are correct.

      Bug details:
      For non zero "mapreduce.task.userlog.limit.kb", the code caches the log contents into memory, waited to be written to file when close() method is called at the end(Ref: ContainerLogAppender.java).
      Explicit call of LogManager.shutdown() from the main() (REF:MRAppMaster.java), ultimately call the *Appender.close().

      Root Cause:
      LogManager.shutdown() is not being called from MRAppMaster.java.
      Similar steps were taken correctly in YarnChild.java.

      Attachments

        1. MAPREDUCE-5694.1.patch
          1 kB
          Mohammad Islam

        Issue Links

          Activity

            People

              kamrul Mohammad Islam
              kamrul Mohammad Islam
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: