Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-6584

Add HiveHBaseTableSnapshotInputFormat

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.14.0
    • HBase Handler
    • Hide
      Hive can now execute queries against HBase table snapshots. This feature is available for any table defined using the HBaseStorageHandler. It requires at least HBase 0.98.3.

      To query against a snapshot instead of the online table, specify the snapshot name via hive.hbase.snapshot.name. The snapshot will be restored into a unique directory under /tmp. This location can be overridden by setting a path via hive.hbase.snapshot.restoredir.
      Show
      Hive can now execute queries against HBase table snapshots. This feature is available for any table defined using the HBaseStorageHandler. It requires at least HBase 0.98.3. To query against a snapshot instead of the online table, specify the snapshot name via hive.hbase.snapshot.name. The snapshot will be restored into a unique directory under /tmp. This location can be overridden by setting a path via hive.hbase.snapshot.restoredir.

    Description

      HBASE-8369 provided mapreduce support for reading from HBase table snapsopts. This allows a MR job to consume a stable, read-only view of an HBase table directly off of HDFS. Bypassing the online region server API provides a nice performance boost for the full scan. HBASE-10642 is backporting that feature to 0.94/0.96 and also adding a mapred implementation. Once that's available, we should add an input format. A follow-on patch could work out how to integrate this functionality into the StorageHandler, similar to how HIVE-6473 integrates the HFileOutputFormat into existing table definitions.

      Attachments

        1. HIVE-6584.0.patch
          21 kB
          Nick Dimiduk
        2. HIVE-6584.1.patch
          21 kB
          Teng Yutong
        3. HIVE-6584.2.patch
          22 kB
          Nick Dimiduk
        4. HIVE-6584.3.patch
          23 kB
          Nick Dimiduk
        5. HIVE-6584.4.patch
          25 kB
          Nick Dimiduk
        6. HIVE-6584.5.patch
          26 kB
          Teng Yutong
        7. HIVE-6584.6.patch
          26 kB
          Teng Yutong
        8. HIVE-6584.7.patch
          46 kB
          Teng Yutong
        9. HIVE-6584.8.patch
          58 kB
          Nick Dimiduk
        10. HIVE-6584.9.patch
          41 kB
          Nick Dimiduk
        11. HIVE-6584.10.patch
          48 kB
          Nick Dimiduk
        12. HIVE-6584.11.patch
          48 kB
          Nick Dimiduk
        13. HIVE-6584.12.patch
          48 kB
          Nick Dimiduk
        14. HIVE-6584.13.patch
          47 kB
          Nick Dimiduk
        15. HIVE-6584.14.patch
          50 kB
          Nick Dimiduk

        Issue Links

          depends upon

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-11137 Add mapred.TableSnapshotInputFormat

          • Major - Major loss of function.
          • Closed
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-7534 remove reflection from HBaseSplit

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HBASE-11555 TableSnapshotRegionSplit should be public

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-11163 Support providing a Scan instance in mapred.TableMapReduceUtil

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-11557 Counter.class should always be added as dependencyJar

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          links to

          Web Link https://reviews.apache.org/r/23824/

          Activity

            People

              ndimiduk Nick Dimiduk
              ndimiduk Nick Dimiduk
              Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: