Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-16907

"INSERT INTO" overwrite old data when destination table encapsulated by backquote

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.1.0, 2.1.1
    • 4.0.0-alpha-1
    • Parser
    • None

    Description

      A way to reproduce:

      create database tdb;
      use tdb;
      create table t1(id int);
      create table t2(id int);
      explain insert into `tdb.t1` select * from t2;
      
      +---------------------------------------------------------------------------------------------------------------------------------------------------+
      |                                                                      Explain                                                                      |
      +---------------------------------------------------------------------------------------------------------------------------------------------------+
      | STAGE DEPENDENCIES:                                                                                                                               |
      |   Stage-1 is a root stage                                                                                                                         |
      |   Stage-6 depends on stages: Stage-1 , consists of Stage-3, Stage-2, Stage-4                                                                      |
      |   Stage-3                                                                                                                                         |
      |   Stage-0 depends on stages: Stage-3, Stage-2, Stage-5                                                                                            |
      |   Stage-2                                                                                                                                         |
      |   Stage-4                                                                                                                                         |
      |   Stage-5 depends on stages: Stage-4                                                                                                              |
      |                                                                                                                                                   |
      | STAGE PLANS:                                                                                                                                      |
      |   Stage: Stage-1                                                                                                                                  |
      |     Map Reduce                                                                                                                                    |
      |       Map Operator Tree:                                                                                                                          |
      |           TableScan                                                                                                                               |
      |             alias: t2                                                                                                                             |
      |             Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE                                                             |
      |             Select Operator                                                                                                                       |
      |               expressions: id (type: int)                                                                                                         |
      |               outputColumnNames: _col0                                                                                                            |
      |               Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE                                                           |
      |               File Output Operator                                                                                                                |
      |                 compressed: false                                                                                                                 |
      |                 Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE                                                         |
      |                 table:                                                                                                                            |
      |                     input format: org.apache.hadoop.hive.ql.io.RCFileInputFormat                                                                  |
      |                     output format: org.apache.hadoop.hive.ql.io.RCFileOutputFormat                                                                |
      |                     serde: org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe                                                                   |
      |                     name: tdb.t1                                                                                                                  |
      |                                                                                                                                                   |
      |   Stage: Stage-6                                                                                                                                  |
      |     Conditional Operator                                                                                                                          |
      |                                                                                                                                                   |
      |   Stage: Stage-3                                                                                                                                  |
      |     Move Operator                                                                                                                                 |
      |       files:                                                                                                                                      |
      |           hdfs directory: true                                                                                                                    |
      |           destination: hdfs://hacluster/user/hive/warehouse/tdb.db/t1/.hive-staging_hive_2017-06-15_15-52-34_017_849305017872068583-1/-ext-10000  |
      |                                                                                                                                                   |
      |   Stage: Stage-0                                                                                                                                  |
      |     Move Operator                                                                                                                                 |
      |       tables:                                                                                                                                     |
      |           replace: true                                                                                                                           |
      |           table:                                                                                                                                  |
      |               input format: org.apache.hadoop.hive.ql.io.RCFileInputFormat                                                                        |
      |               output format: org.apache.hadoop.hive.ql.io.RCFileOutputFormat                                                                      |
      |               serde: org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe                                                                         |
      |               name: tdb.t1                                                                                                                        |
      |                                                                                                                                                   |
      |   Stage: Stage-2                                                                                                                                  |
      |     Merge File Operator                                                                                                                           |
      |       Map Operator Tree:                                                                                                                          |
      |           RCFile Merge Operator                                                                                                                   |
      |       merge level: block                                                                                                                          |
      |       input format: org.apache.hadoop.hive.ql.io.RCFileInputFormat                                                                                |
      |                                                                                                                                                   |
      |   Stage: Stage-4                                                                                                                                  |
      |     Merge File Operator                                                                                                                           |
      |       Map Operator Tree:                                                                                                                          |
      |           RCFile Merge Operator                                                                                                                   |
      |       merge level: block                                                                                                                          |
      |       input format: org.apache.hadoop.hive.ql.io.RCFileInputFormat                                                                                |
      |                                                                                                                                                   |
      |   Stage: Stage-5                                                                                                                                  |
      |     Move Operator                                                                                                                                 |
      |       files:                                                                                                                                      |
      |           hdfs directory: true                                                                                                                    |
      |           destination: hdfs://hacluster/user/hive/warehouse/tdb.db/t1/.hive-staging_hive_2017-06-15_15-52-34_017_849305017872068583-1/-ext-10000  |
      |                                                                                                                                                   |
      +---------------------------------------------------------------------------------------------------------------------------------------------------+
      
      

      Note that 'replace: true' in move operator

      Attachments

        1. HIVE-16907.03.patch
          20 kB
          Zoltan Haindrich
        2. HIVE-16907.03.patch
          20 kB
          Zoltan Haindrich
        3. HIVE-16907.03.patch
          20 kB
          Zoltan Haindrich
        4. HIVE-16907.03.patch
          20 kB
          Zoltan Haindrich
        5. HIVE-16907.02.patch
          18 kB
          Zoltan Haindrich
        6. HIVE-16907.1.patch
          2 kB
          Bing Li

        Issue Links

          Activity

            People

              kgyrtkirk Zoltan Haindrich
              nemon Nemon Lou
              Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: