Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-1609

Support partition filtering in metastore

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.7.0
    • Metastore
    • Reviewed
    • Hide
      Added support for a new listPartitionsByFilter API in HiveMetaStoreClient. This returns the list of partitions matching a specified partition filter. The filter supports "=", "!=", ">", "<", ">=", "<=" and "LIKE" operations on partition keys of type string. "AND" and "OR" logical operations are supported in the filter. So for example, for a table having partition keys country and state, the filter can be 'country = "USA" AND (state = "CA" OR state = "AZ")'
      Show
      Added support for a new listPartitionsByFilter API in HiveMetaStoreClient. This returns the list of partitions matching a specified partition filter. The filter supports "=", "!=", ">", "<", ">=", "<=" and "LIKE" operations on partition keys of type string. "AND" and "OR" logical operations are supported in the filter. So for example, for a table having partition keys country and state, the filter can be 'country = "USA" AND (state = "CA" OR state = "AZ")'

    Description

      WARNING: This patch was subsequently disabled in HIVE-1853 due to stability concerns related to the JDO version upgrade.

      The metastore needs to have support for returning a list of partitions based on user specified filter conditions. This will be useful for tools which need to do partition pruning. Howl is one such use case. The way partition pruning is done during hive query execution need not be changed.

      Attachments

        1. hive_1609_3.patch
          132 kB
          Ajay Kidave
        2. hive_1609_2.patch
          192 kB
          Ajay Kidave
        3. hive_1609.patch
          187 kB
          Ajay Kidave

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-1664 Eclipse build broken

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-1539 Concurrent metastore threading problem

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HCATALOG-347 Pig loads all partitions when it is specifically told not to load all

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HIVE-1853 downgrade JDO version

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HIVE-1862 Revive partition filtering in the Hive MetaStore

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-2048 Expedite Partition Pruning

          • Major - Major loss of function.
          • Open

          Activity

            People

              ajaykidave Ajay Kidave
              ajaykidave Ajay Kidave
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: