Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3962

InverseMapper causes IntWritable type to collected as Text

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.0.0
    • None
    • None
    • None

    Description

      When using the org.apache.hadoop.mapred.lib.InverseMapper, key,values are inversed as expected. For example (Text,IntWritable) will get inversed. However, output key,value only works if you use (Text, Text). Below is an example, where I was chaining 2 jobs:

      Job 1:
      public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable>

      {...}

      Job 2:
      conf.setMapperClass(InverseMapper.class);
      public static class Reduce extends MapReduceBase implements Reducer<IntWritable, Text, Text, IntWritable> {...}

      //I would expect this to work. When I do this, I get
      java.lang.ClassCastException: org.apache.hadoop.io.Text cannot be cast to org.apache.hadoop.io.IntWritable

      To re-inverse my key,values, I had to do this:
      public static class Reduce extends MapReduceBase implements Reducer<Text, Text, Text, Text>

      Notice that in order for the reducer to properly accept key,values, I had to indicate the key as Text.

      Attachments

        Activity

          People

            Unassigned Unassigned
            saadpatel Saad Patel
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated: