Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-8065 Support HDFS encryption functionality on Hive
  3. HIVE-10629

Dropping table in an encrypted zone does not drop warehouse directory

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.1.0
    • 1.2.1
    • Security
    • None

    Description

      Drop table in an encrypted zone removes the table but not its data. The client sees the following on Hive CLI:

      hive> drop table testtbl;
OK
Time taken: 0.158 seconds

      On the Hive Metastore log following error is thrown:

      2015-05-05 08:55:27,665 ERROR [pool-6-thread-142]: hive.log (MetaStoreUtils.java:logAndThrowMetaException(1200)) - Got exception: java.io.IOException Failed to move to trash: hdfs://node-1.example.com:8020/apps/hive/warehouse/encdb1.db/testtbl
java.io.IOException: Failed to move to trash: hdfs://node-1.example.com:8020/apps/hive/warehouse/encdb1.db/testtbl
        at org.apache.hadoop.fs.TrashPolicyDefault.moveToTrash(TrashPolicyDefault.java:160)
        at org.apache.hadoop.fs.Trash.moveToTrash(Trash.java:114)
        at org.apache.hadoop.fs.Trash.moveToAppropriateTrash(Trash.java:95)
        at org.apache.hadoop.hive.shims.Hadoop23Shims.moveToAppropriateTrash(Hadoop23Shims.java:270)
        at org.apache.hadoop.hive.metastore.HiveMetaStoreFsImpl.deleteDir(HiveMetaStoreFsImpl.java:47)
        at org.apache.hadoop.hive.metastore.Warehouse.deleteDir(Warehouse.java:229)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.deleteTableData(HiveMetaStore.java:1584)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table_core(HiveMetaStore.java:1552)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table_with_environment_context(HiveMetaStore.java:1705)
        at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
        at com.sun.proxy.$Proxy13.drop_table_with_environment_context(Unknown Source)
        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$drop_table_with_environment_context.getResult(ThriftHiveMetastore.java:9256)
        ....

      The client should throw the error and maybe fail the drop table call. To delete the table data one currently has to use drop table testtbl purge which basically remove the table data permanently skipping trash.

      Attachments

        1. HIVE-10629.patch
          8 kB
          Eugene Koifman
        2. HIVE-10629.2.patch
          8 kB
          Eugene Koifman
        3. HIVE-10629.3.patch
          8 kB
          Eugene Koifman
        4. HIVE-10629.4.patch
          8 kB
          Eugene Koifman
        5. HIVE-10629.5.patch
          32 kB
          Eugene Koifman

        Issue Links

          is related to

          Sub-task - The sub-task of the issue HIVE-10986 Check of fs.trash.interval in HiveMetaStore should be consistent with Trash.moveToAppropriateTrash()

          • Major - Major loss of function.
          • Open
          relates to

          Sub-task - The sub-task of the issue HIVE-10747 Enable the cleanup of side effect for the Encryption related qfile test

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HIVE-10717 Fix failed qtest encryption_insert_partition_static test in Jenkin

          • Major - Major loss of function.
          • Closed
          requires

          Bug - A problem which impairs or prevents the functions of the product. HIVE-10978 Document fs.trash.interval wrt Hive and HDFS Encryption

          • Critical - Crashes, loss of data, severe memory leak.
          • Open

          Activity

            People

              ekoifman Eugene Koifman
              deepesh Deepesh Khandelwal
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: