Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-12411

Remove counter based stats collection mechanism

    XMLWordPrintableJSON

Details

    • Task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.2.0, 1.2.1
    • 2.0.0
    • Statistics

    Description

      Following HIVE-12005, HIVE-12164, we have removed jdbc and hbase stats collection mechanism. Now we are targeting counter based stats collection mechanism. The main advantages are as follows (1) counter based stats has limitation on the length of the counter itself, if it is too long, MD5 will be applied. (2) when there are a large number of partitions and columns, we need to create a large number of counters in memory. This will put a heavy load on the M/R AM or Tez AM etc. FS based stats will do a better job.

      Attachments

        1. HIVE-12411.02.patch
          118 kB
          Pengcheng Xiong
        2. HIVE-12411.01.patch
          112 kB
          Pengcheng Xiong

        Issue Links

          Activity

            People

              pxiong Pengcheng Xiong
              pxiong Pengcheng Xiong
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: