Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-19157

Assert that Insert into Druid Table fails if the publishing of metadata by HS2 fails

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 3.1.0, 3.0.0
    • None
    • None

    Description

      The usual work flow of loading Data into Druid relies on the fact that HS2 is able to load Segments metadata from HDFS that are produced by LLAP/TEZ works.
      In some cases where HS2 is not able to perform `ls` on the HDFS path the insert into query will return success and will not insert any data.
      This bug was introduced at function

       org.apache.hadoop.hive.druid.DruidStorageHandlerUtils#getCreatedSegments


      when we added feature to allow create empty tables.

       try {
            fss = fs.listStatus(taskDir);
          } catch (FileNotFoundException e) {
            // This is a CREATE TABLE statement or query executed for CTAS/INSERT
            // did not produce any result. We do not need to do anything, this is
            // expected behavior.
            return publishedSegmentsBuilder.build();
          }
      

      Am still looking for the way to fix this, jcamachorodriguez/ashutoshc any idea what is the best way to detect that it is an empty create table statement?

      Attachments

        1. HIVE-19157.patch
          9 kB
          Slim Bouguerra
        2. HIVE-19157.4.patch
          17 kB
          Slim Bouguerra
        3. HIVE-19157.3.patch
          10 kB
          Slim Bouguerra
        4. HIVE-19157.2.patch
          10 kB
          Slim Bouguerra

        Issue Links

          Activity

            People

              bslim Slim Bouguerra
              bslim Slim Bouguerra
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: