Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-33537 Hive Metastore filter pushdown improvement
  3. SPARK-33584

Partition predicate pushdown into Hive metastore support cast string type to date type

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • 3.1.0
    • None
    • SQL
    • None

    Description

      spark.sql("create table t1(id string) partitioned by (part string) stored as parquet")
spark.sql("insert into t1 values('1', '2019-01-01')")
spark.sql("insert into t1 values('2', '2019-01-02')")
spark.sql("select * from t1 where  part = date '2019-01-01' ").show

      We can pushdown {{cast(part as date) = date '2019-01-01' }} to Hive metastore.

      Attachments

        Activity

          People

            yumwang Yuming Wang
            yumwang Yuming Wang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: