Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-27143

Add hbase-unsafe as a dependency for a MR job triggered by hbase shell

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      After HBASE-26523, the use of hbase-thirdparty 4.1.0 introduced an new dependency of hbase-unsafe that wasn't added as part of the MR job triggered by hbase-shell. As such, class not found exception was being emitted when executing in distributed mode.

      22/06/21 04:57:15 INFO mapreduce.Job: Task Id : attempt_1655784404826_0011_m_000000_0, Status : FAILED
Error: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.unsafe.HBasePlatformDependent
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
	at org.apache.hadoop.hbase.util.Bytes.<clinit>(Bytes.java:129)
	at org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator$GeneratorMapper.setup(IntegrationTestBigLinkedList.java:455)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)

      This task will add it back as part of the job configuration handled by hbase internally in addHBaseDependencyJars

      Attachments

        Issue Links

          is related to

          Task - A task that needs to be done. HBASE-26523 Upgrade hbase-thirdparty dependency to 4.0.1

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved
          links to

          Web Link GitHub Pull Request #4554

          Activity

            People

              taklwu Tak-Lon (Stephen) Wu
              taklwu Tak-Lon (Stephen) Wu
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: