Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-39786 Fix performance gap and inconsistent behavior between using Scala 2.12 and Scala 2.13
  3. SPARK-39652

For CoalescedRDDBenchmark, the test results using Scala 2.13 are slower than Scala 2.12`, the test results using Scala 2.13 are slower than Scala 2.12

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.4.0
    • None
    • Spark Core
    • None

    Description

      Typical result are as follows:

      2.12.16

      OpenJDK 64-Bit Server VM 1.8.0_332-b09 on Linux 5.13.0-1031-azure
      Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
      Coalesced RDD:                                Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
      ----------------------------------------------------------------------------------------------------------------------------
      Coalesce Num Partitions: 500 Num Hosts: 1               659            671          19          0.2        6588.2       0.4X
      Coalesce Num Partitions: 1000 Num Hosts: 1             1111           1133          25          0.1       11114.3       0.3X
      Coalesce Num Partitions: 5000 Num Hosts: 1             4573           4580          12          0.0       45727.1       0.1X
      Coalesce Num Partitions: 10000 Num Hosts: 1            8949           9107         240          0.0       89494.7       0.0X

       

      2.13.8

      OpenJDK 64-Bit Server VM 1.8.0_332-b09 on Linux 5.13.0-1031-azure
      Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
      Coalesced RDD:                                Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
      ----------------------------------------------------------------------------------------------------------------------------
      Coalesce Num Partitions: 500 Num Hosts: 1               964           1131         284          0.1        9643.3       0.3X
      Coalesce Num Partitions: 1000 Num Hosts: 1             1732           1742          10          0.1       17318.2       0.2X
      Coalesce Num Partitions: 5000 Num Hosts: 1             7534           7539           4          0.0       75339.8       0.0X
      Coalesce Num Partitions: 10000 Num Hosts: 1           14524          14565          37          0.0      145245.0       0.0X 

       

      From the analysis of jfr results, the hot path is `ArrayBuffer.min`

      https://github.com/apache/spark/blob/ded5981823ac8e8e9339291415d9828dfcc6e062/core/src/main/scala/org/apache/spark/rdd/CoalescedRDD.scala#L224-L225

       

      def getLeastGroupHash(key: String): Option[PartitionGroup] =
        groupHash.get(key).filter(_.nonEmpty).map(_.min) 

       

      then I write a new simple bench to test `ArrayBuffer.min` and `ArrayBuffer.max`:

      private def min(numIters: Int, bufferSize: Int, loops: Int): Unit = {
        val benchmark = new Benchmark("Array Buffer", loops, output = output)
        val buffer = new mutable.ArrayBuffer[Int]()
        (0 until bufferSize).foreach(i => buffer += i)
        benchmark.addCase(s"buffer min with $bufferSize items", numIters) { _ =>
          (0 until loops).foreach(_ => buffer.min)
        }
        benchmark.addCase(s"buffer sorted head with $bufferSize items", numIters) { _ =>
          (0 until loops).foreach(_ => buffer.sorted.head)
        }
        benchmark.addCase(s"buffer max with $bufferSize items", numIters) { _ =>
          (0 until loops).foreach(_ => buffer.max)
        }
        benchmark.addCase(s"buffer sorted last with $bufferSize items", numIters) { _ =>
          (0 until loops).foreach(_ => buffer.sorted.last)
        }
        benchmark.run()
      }
      
      override def runBenchmarkSuite(mainArgs: Array[String]): Unit = {
        val numIters = 3
        val loops = 20000
        runBenchmark("ArrayBuffer min and max") {
          Seq(1000, 10000, 100000).foreach { bufferSize =>
            min(numIters, bufferSize, loops)
          }
        }
      } 

       

      The result as follows:

      Scala 2.12

       

      OpenJDK 64-Bit Server VM 1.8.0_332-b09 on Linux 5.13.0-1031-azure
      Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
      Array Buffer:                             Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
      ------------------------------------------------------------------------------------------------------------------------
      buffer min with 1000 items                          121            121           0          0.2        6043.9       1.0X
      buffer max with 1000 items                          134            134           0          0.1        6705.5       0.9XOpenJDK 64-Bit Server VM 1.8.0_332-b09 on Linux 5.13.0-1031-azure
      Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
      Array Buffer:                             Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
      ------------------------------------------------------------------------------------------------------------------------
      buffer min with 10000 items                        1272           1272           0          0.0       63601.9       1.0X
      buffer max with 10000 items                        1339           1340           0          0.0       66972.2       0.9XOpenJDK 64-Bit Server VM 1.8.0_332-b09 on Linux 5.13.0-1031-azure
      Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
      Array Buffer:                             Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
      ------------------------------------------------------------------------------------------------------------------------
      buffer min with 100000 items                      12785          12788           3          0.0      639232.4       1.0X
      buffer max with 100000 items                      13433          13434           1          0.0      671654.2       1.0X 

       

      Scala 2.13

      OpenJDK 64-Bit Server VM 1.8.0_332-b09 on Linux 5.13.0-1031-azure
      Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
      Array Buffer:                             Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
      ------------------------------------------------------------------------------------------------------------------------
      buffer min with 1000 items                          273            273           0          0.1       13646.5       1.0X
      buffer max with 1000 items                          162            162           0          0.1        8076.9       1.7XOpenJDK 64-Bit Server VM 1.8.0_332-b09 on Linux 5.13.0-1031-azure
      Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
      Array Buffer:                             Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
      ------------------------------------------------------------------------------------------------------------------------
      buffer min with 10000 items                        1676           1676           0          0.0       83782.1       1.0X
      buffer max with 10000 items                        1610           1610           0          0.0       80485.5       1.0XOpenJDK 64-Bit Server VM 1.8.0_332-b09 on Linux 5.13.0-1031-azure
      Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
      Array Buffer:                             Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
      ------------------------------------------------------------------------------------------------------------------------
      buffer min with 100000 items                      16849          16856          10          0.0      842469.3       1.0X
      buffer max with 100000 items                      15513          15515           4          0.0      775641.0       1.1X

       

      Already submit a issue to scala:  https://github.com/scala/bug/issues/12614 

      Attachments

        Activity

          People

            Unassigned Unassigned
            LuciferYang Yang Jie
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: