Details
-
Umbrella
-
Status: Open
-
Major
-
Resolution: Unresolved
-
3.4.0
-
None
-
None
Description
After SPARK-39576, we can run benchmark GitHub Actions to to generate Scala 2.13 benchmark results, so that we can compare with the result of Scala 2.12 one by one to find the performance gap and fix them.
But if the performance gap is bug of Scala 2.13, we should report it to the Scala community and try to find a workaround way in Spark. If the workaround way not exists due we need to support two Scala versions now, we can left TODO and fix them after Spark only supports Scala 2.13.
On the other hand, by comparing the UTs or run Scala 2.13 compiled Spark in the production environment, we may find some behavioral differences, they can also be classified as subTasks of the current umbrella.