Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10481

return application id when submitting job

    XMLWordPrintableJSON

Details

    • Wish
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 3.2.1
    • None
    • yarn

    Description

      Hello team, 

      Submitting jobs through 'yarn jar mainClass args' is a general task, however, we have to parse the current application id or tracking URL of the submitted job from the INFO logs, such as 

      20/11/03 00:39:13 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm2
      20/11/03 00:39:13 INFO input.FileInputFormat: Total input paths to process : 16
      20/11/03 00:39:13 INFO mapreduce.JobSubmitter: number of splits:16
      20/11/03 00:39:13 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1601247375834_530832
      20/11/03 00:39:14 INFO impl.YarnClientImpl: Submitted application application_1601247375834_530832
      20/11/03 00:39:14 INFO mapreduce.Job: The url to track the job: https://xxxxxxxx:50030/proxy/application_1601247375834_530832/
      20/11/03 00:39:14 INFO mapreduce.Job: Running job: job_1601247375834_530832
      20/11/03 00:39:26 INFO mapreduce.Job: Job job_1601247375834_530832 running in uber mode : false
      20/11/03 00:39:26 INFO mapreduce.Job: map 0% reduce 0%
      20/11/03 00:39:34 INFO mapreduce.Job: map 13% reduce 0%
      20/11/03 00:39:35 INFO mapreduce.Job: map 50% reduce 0%
      20/11/03 00:39:36 INFO mapreduce.Job: map 63% reduce 0%
      20/11/03 00:39:37 INFO mapreduce.Job: map 88% reduce 0%
      20/11/03 00:39:38 INFO mapreduce.Job: map 100% reduce 0%
      20/11/03 00:39:47 INFO mapreduce.Job: map 100% reduce 100%

       

      So my question is that, is there a convenient way to retrieve the application for the submitted job?

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              gabry.wu gabrywu
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: