Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3245

Provide ability to persist running jobs (extend HADOOP-1876)

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.19.0
    • None
    • None
    • Incompatible change, Reviewed
    • Introduced recovery of jobs when JobTracker restarts. This facility is off by default. Introduced config parameters mapred.jobtracker.restart.recover, mapred.jobtracker.job.history.block.size, and mapred.jobtracker.job.history.buffer.size.

    Description

      This could probably extend the work done in HADOOP-1876. This feature can be applied for things like jobs being able to survive jobtracker restarts.

      Attachments

        1. HADOOP-3245-v2.5.patch
          53 kB
          Amar Kamat
        2. HADOOP-3245-v2.6.5.patch
          84 kB
          Amar Kamat
        3. HADOOP-3245-v2.6.9.patch
          84 kB
          Amar Kamat
        4. HADOOP-3245-v4.1.patch
          115 kB
          Amar Kamat
        5. HADOOP-3245-v5.13.patch
          187 kB
          Amar Kamat
        6. HADOOP-3245-v5.14.patch
          170 kB
          Amar Kamat
        7. HADOOP-3245-v5.26.patch
          188 kB
          Amar Kamat
        8. HADOOP-3245-v5.30-nolog.patch
          160 kB
          Amar Kamat
        9. HADOOP-3245-v5.31.3-nolog.patch
          149 kB
          Amar Kamat
        10. HADOOP-3245-v5.33.1.patch
          175 kB
          Amar Kamat
        11. HADOOP-3245-v5.35.3-no-log.patch
          170 kB
          Amar Kamat
        12. HADOOP-3245-v5.36.1-no-log.patch
          176 kB
          Amar Kamat
        13. HADOOP-3245-v5.36.2-no-log.patch
          176 kB
          Amar Kamat
        14. HADOOP-3245-v5.36.3-no-log.patch
          177 kB
          Amar Kamat
        15. HADOOP-3245-v5.36.3-no-log.patch
          176 kB
          Amar Kamat
        16. HADOOP-3245-v5.36-no-log.patch
          169 kB
          Amar Kamat

        Issue Links

          blocks

          Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-277 Job history counters should be avaible on the UI.

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed
          depends upon

          Bug - A problem which impairs or prevents the functions of the product. HDFS-53 RawLocalFileSystem.listStatus() should not throw FileNotFound exception if a file goes missing while the list is being computed

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-7 Reduce progress in the Reducer crosses 1

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-2403 JobHistory log files contain data that cannot be parsed by org.apache.hadoop.mapred.JobHistory

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-3970 Counters written to the job history cannot be recovered back

          • Major - Major loss of function.
          • Closed
          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-3590 Null pointer exception in JobTracker when the task tracker is not yet resolved

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-3947 TaskTrackers fail to connect back upon a re-init action

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-3641 Exception while killing the JobTracker

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-4112 Got ArrayOutOfBound exception while analyzing the job history

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-3780 JobTracker should synchronously resolve the tasktracker's network location when the tracker registers

          • Major - Major loss of function.
          • Closed
          is depended upon by

          New Feature - A new feature of the product, which has yet to be developed. MAPREDUCE-225 Fault tolerant Hadoop Job Tracker

          • Major - Major loss of function.
          • Resolved
          is part of

          New Feature - A new feature of the product, which has yet to be developed. HADOOP-3444 Implementing a Resource Manager (V1) for Hadoop

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-5436 job history directory grows without bound, locks up job tracker on new job submission

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-4220 Job Restart tests take 10 minutes, can time out very easily

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-3771 JobClient.runJob() should not kill the job on IOExceptions

          • Major - Major loss of function.
          • Closed

          Activity

            People

              amar_kamat Amar Kamat
              ddas Devaraj Das
              Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: