Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-7292 Hive on Spark
  3. HIVE-8406

Research on skewed join [Spark Branch]

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Done
    • None
    • None
    • Spark
    • None

    Description

      Research on how to handle skewed join for hive on spark. Here is original hive's design doc for skewed join, https://cwiki.apache.org/confluence/display/Hive/Skewed+Join+Optimization.

      Attachments

        1. Skew join background.pdf
          596 kB
          Rui Li

        Issue Links

          depends upon

          Sub-task - The sub-task of the issue HIVE-7613 Research optimization of auto convert join to map join [Spark branch]

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          incorporates

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-8958 Make sure map join tasks created by runtime skew join can fit into memory [Spark Branch]

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-8535 Enable compile time skew join optimization for spark [Spark Branch]

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-8536 Enable SkewJoinResolver for spark [Spark Branch]

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-8913 Make SparkMapJoinResolver handle runtime skew join [Spark Branch]

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-9097 Support runtime skew join for more queries [Spark Branch]

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              lirui Rui Li
              xuefuz Xuefu Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: