Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6980

Add support of ARM64 hardware crc instructions for nativetask

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Patch Available
    • Minor
    • Resolution: Unresolved
    • 3.0.0-alpha3
    • None
    • nativetask, performance
    • None
    • ARM64 platform with CRC extension

    Description

      CRC instructions are introduced since ARM64V8 ISA. With this extension Arm supports CRC32(0x04C11DB7) and CRC32C(0x1EDC6F41) checksum hardware computation.
      This patch enables support on such arm64 platform, similiar as HADOOP-11660.
      Benchmark is done on a A57 platform using nttest (hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/target/native/test/nttest), and the result is as following:
      Before (sw crc):
      [ RUN ] Perf.CRC
      17/10/10 03:05:18 INFO CRC time: 0.10112s size: 100.000M speed: 988.92M/s
      17/10/10 03:05:18 INFO CRC32C time: 0.10043s size: 100.000M speed: 995.74M/s
      [ OK ] Perf.CRC (216 ms)

      After (hw crc):
      [ RUN ] Perf.CRC
      17/10/11 05:06:27 INFO CRC time: 0.03173s size: 100.000M speed: 3151.14M/s
      17/10/11 05:06:27 INFO CRC32C time: 0.03146s size: 100.000M speed: 3178.97M/s
      [ OK ] Perf.CRC (77 ms)

      Attachments

        1. MAPREDUCE-6980.0.patch
          13 kB
          Jun He

        Activity

          People

            Unassigned Unassigned
            jun.he Jun He
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: