Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-37063 SQL Adaptive Query Execution QA: Phase 2
  3. SPARK-35540

Make config maxShuffledHashJoinLocalMapThreshold fallback to advisoryPartitionSizeInBytes

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: In Progress
    • Major
    • Resolution: Unresolved
    • 3.2.0
    • None
    • SQL
    • None

    Description

      Make config `spark.sql.adaptive.maxShuffledHashJoinLocalMapThreshold` fallback to `spark.sql.adaptive.advisoryPartitionSizeInBytes` so that we can convert SMJ to SHJ in AQE by default.

      Attachments

        Activity

          People

            Unassigned Unassigned
            ulysses XiDuo You
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: