Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9542

Fix LogsCLI guessAppOwner ignores custom file format suffix

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 3.2.0, 3.1.2
    • 3.2.1, 3.1.3
    • log-aggregation
    • None

    Description

      LogsCLI guessAppOwner ignores custom file format suffix yarn.log-aggregation.%s.remote-app-log-dir-suffix / Default IndexedFileController Suffix (

      {yarn.nodemanager.remote-app-log-dir-suffix}

      -ifile or logs-ifile). It considers only yarn.nodemanager.remote-app-log-dir-suffix or default logs.

      Repro:

      
      yarn-site.xml
      
      yarn.log-aggregation.file-formats ifile
      yarn.log-aggregation.file-controller.ifile.class org.apache.hadoop.yarn.logaggregation.filecontroller.ifile.LogAggregationIndexedFileController
      yarn.log-aggregation.ifile.remote-app-log-dir app-logs
      yarn.resourcemanager.connect.max-wait.ms 1000
      
      core-site.xml:
      
      ipc.client.connect.max.retries 3
      ipc.client.connect.retry.interval 10
      
      Run a Job with above configs and Stop the RM.
      
      [ambari-qa@yarn-ats-1 ~]$ yarn logs -applicationId application_1557482389195_0001
      2019-05-10 10:03:58,215 INFO client.RMProxy: Connecting to ResourceManager at yarn-ats-1/172.26.81.91:8050
      Unable to get ApplicationState. Attempting to fetch logs directly from the filesystem.
      Can not find the appOwner. Please specify the correct appOwner
      Could not locate application logs for application_1557482389195_0001
      
      
      [ambari-qa@yarn-ats-1 ~]$ hadoop fs -ls /app-logs/ambari-qa/logs-ifile/application_1557482389195_0001
      Found 1 items
      -rw-r-----   3 ambari-qa supergroup      18058 2019-05-10 10:01 /app-logs/ambari-qa/logs-ifile/application_1557482389195_0001/yarn-ats-1_45454
      
      

      Attachments

        1. YARN-9542-branch-3.2.002.patch
          28 kB
          Prabhu Joseph
        2. YARN-9542-branch-3.2.001.patch
          28 kB
          Prabhu Joseph
        3. YARN-9542-001.patch
          9 kB
          Prabhu Joseph

        Issue Links

          Activity

            People

              prabhujoseph Prabhu Joseph
              prabhujoseph Prabhu Joseph
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: