Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-2715

Probe filter are not pushed to scan for joins on partitioned column

    XMLWordPrintableJSON

Details

    Description

      Summary shows that no probe filters are pushed to the store_sales scan.

      Query

      select 
          count(*)
      from
          store_sales,
          date_dim d1
      where
          d1.d_date_sk = ss_sold_date
              and d_dom = 2
              and d1.d_quarter_name = '2000Q1'
      

      Summary

      Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail
      06:AGGREGATE 1 148.39us 148.39us 1 1 16.00 KB -1 B FINALIZE
      05:EXCHANGE 1 16.63us 16.63us 1 1 0 B -1 B UNPARTITIONED
      03:AGGREGATE 1 0ns 0ns 1 1 3.86 MB 10.00 MB  
      02:HASH JOIN 1 21.28s 21.28s 0 183.59K 2.02 MB 2.00 GB INNER JOIN, BROADCAST
      --04:EXCHANGE 1 22.15us 22.15us 3 -1 0 B 0 B BROADCAST
      01:SCAN HDFS 1 75.22ms 75.22ms 3 -1 1.97 MB 16.00 MB tpcds_parquet.date_dim d1
      00:SCAN HDFS 1 316.16ms 316.16ms 183.59K 183.59K 568.00 KB 16.00 MB tpcds_parquet.store_sales_p

      While for store_sales x store probe filters are pushed to the store_sales scan

      select count(*)
         from  store_sales
              ,store
         where ss_store_sk = s_store_sk
         and   s_store_id="AAAAAAAABAAAAAAA";
      
      Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail
      06:AGGREGATE 1 188.66us 188.66us 1 1 16.00 KB -1 B FINALIZE
      05:EXCHANGE 1 26.11us 26.11us 1 1 0 B -1 B UNPARTITIONED
      03:AGGREGATE 1 3.23ms 3.23ms 1 1 164.00 KB 10.00 MB  
      02:HASH JOIN 1 24.97ms 24.97ms 30.99K 183.59K 2.02 MB 2.00 GB INNER JOIN, BROADCAST
      --04:EXCHANGE 1 29.18us 29.18us 1 -1 0 B 0 B BROADCAST
      01:SCAN HDFS 1 3.10s 3.10s 1 -1 62.00 KB 16.00 MB tpcds_parquet.store
      00:SCAN HDFS 1 290.67ms 290.67ms 34.71K 183.59K 160.00 KB 16.00 MB tpcds_parquet.store_sales

      Attachments

        Issue Links

          Activity

            People

              henryr Henry Robinson
              mmokhtar Mostafa Mokhtar
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: