Details
-
Documentation
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
3.0.0, 3.0.1
-
None
Description
In the 'Optimizing Skew Join' section of the following two pages:
- https://spark.apache.org/docs/3.0.0/sql-performance-tuning.html
- https://spark.apache.org/docs/3.0.1/sql-performance-tuning.html
The configuration 'spark.sql.adaptive.skewedPartitionThresholdInBytes' should be changed to 'spark.sql.adaptive.skewJoin.skewedPartitionThresholdInBytes', The former is missing the 'skewJoin'.
Attachments
Issue Links
- links to