Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-22386 Data Source V2 improvements
  3. SPARK-37262

Not log empty aggregate and group by in JDBCScan

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 3.2.0, 3.3.0
    • 3.3.0
    • SQL
    • None

    Description

      Address the comment https://github.com/apache/spark/pull/34451#discussion_r740220800

      Current behavior:

      Scan org.apache.spark.sql.execution.datasources.v2.jdbc.JDBCScan$$anon$1@72e75786 [NAME#1,SALARY#2] PushedAggregates: [], PushedFilters: [IsNotNull(SALARY), GreaterThan(SALARY,100.00)], PushedGroupby: [], ReadSchema: struct<NAME:string,SALARY:decimal(20,2)>

      After the fix, it will be

      Scan org.apache.spark.sql.execution.datasources.v2.jdbc.JDBCScan$$anon$1@72e75786 [NAME#1,SALARY#2] PushedFilters: [IsNotNull(SALARY), GreaterThan(SALARY,100.00)], ReadSchema: struct<NAME:string,SALARY:decimal(20,2)>

      Attachments

        Activity

          People

            huaxingao Huaxin Gao
            huaxingao Huaxin Gao
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: