Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-8084 Sundry mapreduce improvements
  3. HBASE-7934

TableMapReduceUtil doesn't include all dependency jars in new modular build

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.95.0, 0.95.2
    • 0.98.0, 0.95.0
    • mapreduce
    • None
    • Reviewed

    Description

      From post to hbase-dev:

      I'm seeing ClassNotFoundExceptions when running ImportTsv against trunk. Presumably the old instructions for classpath assembly need an update. I guess I could create a hadoop jar that ships the classpath entries in lib, but IMHO that's not a viable solution for users who want to run one of the jobs we distribute. Am I missing something?

      For you reference:

      $ HADOOP_CLASSPATH=$(./bin/hbase classpath) hadoop jar ./hbase-server/target/hbase-server-0.95-SNAPSHOT.jar importtsv -Dimporttsv.columns=HBASE_ROW_KEY,d:c1,d:c2 -Dimporttsv.bulk.output=hdfs:///tmp/hfiles simple hdfs:///tmp/simple.tsv

...

Error: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.protobuf.generated.ClientProtos$Mutate$MutateType
	at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
	at org.apache.hadoop.hbase.mapreduce.MutationSerialization$MutationSerializer.serialize(MutationSerialization.java:87)
	at org.apache.hadoop.hbase.mapreduce.MutationSerialization$MutationSerializer.serialize(MutationSerialization.java:70)
	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1069)
	at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)
	at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
	at org.apache.hadoop.hbase.mapreduce.TsvImporterMapper.map(TsvImporterMapper.java:151)
	at org.apache.hadoop.hbase.mapreduce.TsvImporterMapper.map(TsvImporterMapper.java:37)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
	at org.apache.hadoop.mapred.Child.main(Child.java:249)

      Attachments

        Issue Links

          is depended upon by

          Sub-task - The sub-task of the issue HBASE-7747 Import tools should use a combiner to merge Puts

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          is related to

          Sub-task - The sub-task of the issue HBASE-7938 Add integration test for ImportTsv/LoadIncrementalHFiles workflow

          • Major - Major loss of function.
          • Closed
          relates to

          Task - A task that needs to be done. HBASE-7936 Undo prefix-tree module as dependency for mapreduce and for DataBlockEncoding

          • Major - Major loss of function.
          • Closed

          Activity

            People

              ndimiduk Nick Dimiduk
              ndimiduk Nick Dimiduk
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: