Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-14405

Fix performance regression due to incorrect use of DataChecksum

    XMLWordPrintableJSON

Details

    • Reviewed
    • Patch

    Description

      Recently I have upgraded my Hadoop version from 2.6 to 3.0, and I find that the write performance decreased by 13%. After some days comparative analysis, It's seems introduced by HADOOP-10865.

      Since James Thomas have done the work that native checksum can run against byte[] arrays instead of just against byte buffers, we may use native method preferential because it runs faster than others.

      szetszwo and iwasakims could you take a look at this to see if it make bad effect on your benchmark test? tlipcon could you help to see if I have make mistakes in this patch?
      thanks!

      Attachments

        1. HDFS-11765.patch
          2 kB
          LiXin Ge

        Issue Links

          Activity

            People

              GeLiXin LiXin Ge
              GeLiXin LiXin Ge
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: