Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3697

Hadoop Counters API limits Oozie's working across different hadoop versions

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 0.23.1
    • 0.23.1
    • mrv2
    • None

    Description

      Oozie uses Hadoop Counters API, by invoking Counters.getGroup(). However, in
      hadoop 23, org.apache.hadoop.mapred.Counters does not implement getGroup(). Its
      parent class AbstractCounters implements it. This is different from hadoop20X.
      As a result, Oozie compiled with either hadoop version does not work with the
      other version.
      A specific scenario, Oozie compiled with .23 and run against 205, does not
      update job status owing to a Counters API exception.

      Will explicit re-compilation against the relevant hadoop jars be required each
      time? This will prevent launching a uniform Oozie version across different
      clusters.

      Attachments

        1. MAPREDUCE-3697.patch
          39 kB
          Mahadev Konar
        2. MAPREDUCE-3697.patch
          39 kB
          Mahadev Konar
        3. MAPREDUCE-3697-facade.patch
          6 kB
          Alejandro Abdelnur
        4. MAPREDUCE-3697-facadeV2.patch
          6 kB
          Alejandro Abdelnur
        5. MAPREDUCE-3697-facadeV3.patch
          10 kB
          Alejandro Abdelnur
        6. MAPREDUCE-3697.patch
          23 kB
          Mahadev Konar
        7. MAPREDUCE-3697.patch
          28 kB
          Mahadev Konar
        8. MAPREDUCE-3697.patch
          28 kB
          Mahadev Konar
        9. apicheck-hadoop-1.0.0-0.24.0-SNAPSHOT.sh
          1 kB
          Thomas White
        10. MAPREDUCE-3697.patch
          31 kB
          Mahadev Konar

        Issue Links

          is depended upon by

          Bug - A problem which impairs or prevents the functions of the product. HIVE-2741 Single binary built against 0.20 and 0.23, does not work against 0.23 clusters.

          • Major - Major loss of function.
          • Resolved
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-7738 Document incompatible API changes between 0.20.20x and 0.23.0 release

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-5055 Binary compatibility for the method Reporter.incrCounter(String group, String counter, long amount) between hadoop-1.x and hadoop-2.x

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              mahadev Mahadev Konar
              johnvijoe John George
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: