Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-10362

Support Type check/conversion in dynamic partition column

    XMLWordPrintableJSON

Details

    Description

      There are quite a lot of issues associated with the non-noramlized or type-mismatched values for partition column. Hive has many ways to introduce such problematic data. HIVE-10307 mainly provides the support to type check/convert/normalize the partition column value in static partition specification. This JIRA tries to deal with the partition column type in dynamic partition insert. Currently any data can be inserted as a partition column value as long as it is quoted as a string. For example,
      create table dynparttypechecknum (key int, value string) partitioned by (part int);
      insert into dynparttypechecknum partition (part) select key, value, '00001' from src limit 1;
      show partitions dynparttypechecknum;
      –
      part=00001

      The partition column value is non-normalized int 00001. It causes some unnecessary problems such as integer partition column JDO filter pushdown (see HIVE-6052) and others like HIVE-10210.

      Attachments

        1. HIVE-10362.01.patch
          3 kB
          Karen Coppage
        2. HIVE-10362.02.patch
          8 kB
          Karen Coppage
        3. HIVE-10362.03.patch
          9 kB
          Karen Coppage
        4. HIVE-10362.04.patch
          11 kB
          Karen Coppage
        5. HIVE-10362.04.patch
          11 kB
          Karen Coppage
        6. HIVE-10362.06.patch
          11 kB
          Karen Coppage
        7. HIVE-10362.07.patch
          11 kB
          Karen Coppage
        8. HIVE-10362.07.patch
          11 kB
          Karen Coppage
        9. HIVE-10362.08.patch
          9 kB
          Karen Coppage
        10. HIVE-10362.09.patch
          126 kB
          Karen Coppage
        11. HIVE-10362.10.patch
          127 kB
          Karen Coppage
        12. HIVE-10362.11.patch
          99 kB
          Karen Coppage
        13. HIVE-10362.12.patch
          106 kB
          Karen Coppage
        14. HIVE-10362.12.patch
          106 kB
          Karen Coppage
        15. HIVE-10362.12.patch
          106 kB
          Karen Coppage
        16. HIVE-10362.12.patch
          106 kB
          Karen Coppage
        17. HIVE-10362.12.patch
          106 kB
          Karen Coppage

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. HIVE-10210 Compute partition column stats fails when partition value is zero-leading integer

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-10210 Compute partition column stats fails when partition value is zero-leading integer

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-3976 Handle partition-key values with multiple synonymous string representations created in Hive.

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HIVE-23358 MSCK REPAIR should remove all insignificant zeroes from numeric partition values

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          links to

          Web Link Review board

          Activity

            People

              klcopp Karen Coppage
              ctang Chaoyu Tang
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: