Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-24524

LLAP ShuffleHandler: upgrade to Netty4 and remove Netty3 dependency from hive where it's possible

    XMLWordPrintableJSON

Details

    Description

      Tez already has a WIP patch for upgrading its shuffle handler to netty4. Netty4 is a possible performance improvement compared to Netty3. However, the refactor is not trivial, TEZ-4157 covers that more or less (the code bases are very similar).

      Background:
      netty4 migration guideline: https://netty.io/wiki/new-and-noteworthy-in-4.0.html
      articles of possible performance improvement:
      https://blog.twitter.com/engineering/en_us/a/2013/netty-4-at-twitter-reduced-gc-overhead.html
      https://developer.squareup.com/blog/upgrading-a-reverse-proxy-from-netty-3-to-4/

      some other notes: Netty3 is EOL since 2016:
      https://netty.io/news/2016/06/29/3-10-6-Final.html

      Attachments

        1. dep.log
          866 kB
          László Bodor

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-25013 Remove Netty3 dependency from druid-handler once Druid upgrades fully to Netty4

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. TEZ-4157 ShuffleHandler: upgrade to Netty4 and remove Netty3 dependency from tez

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HIVE-18574 LLAP: Ship netty3 as part of LLAP install tarball

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-11219 [Umbrella] Upgrade to netty 4

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #1778

          Activity

            People

              abstractdog László Bodor
              abstractdog László Bodor
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 50m
                  2h 50m