Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-20948

Eliminate file rename in compactor

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 4.0.0
    • None
    • Transactions
    • None

    Description

      Once HIVE-20823 is committed, we should investigate if it's possible to have compactor write directly to base_x_cZ or delta_x_y_cZ.  

      For query based compaction: can we control location of temp table dir?  We support external temp tables so this may work but we'd need to have non-acid insert create files with bucket_xxxxx names.

       

      For MR/Tez/LLAP based (should this be done at all?), need to figure out how retries of tasks will work.  Just like we currently generate an MR job to compact, we should be able to generate a Tez job.

       

      Attachments

        1. HIVE-20948.01.patch
          40 kB
          László Pintér
        2. HIVE-20948.02.patch
          43 kB
          László Pintér
        3. HIVE-20948.03.patch
          43 kB
          László Pintér
        4. HIVE-20948.04.patch
          43 kB
          László Pintér
        5. HIVE-20948.05.patch
          41 kB
          László Pintér
        6. HIVE-20948.06.patch
          41 kB
          László Pintér
        7. HIVE-20948.07.patch
          41 kB
          László Pintér

        Issue Links

          Activity

            People

              lpinter László Pintér
              ekoifman Eugene Koifman
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: