Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-33507 Improve and fix cache behavior in v1 and v2
  3. SPARK-33950

ALTER TABLE .. DROP PARTITION doesn't refresh cache

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.1, 3.1.0, 3.2.0
    • 3.0.2, 3.1.0, 3.2.0
    • SQL

    Description

      Here is the example to reproduce the issue:

      spark-sql> CREATE TABLE tbl1 (col0 int, part0 int) USING parquet PARTITIONED BY (part0);
spark-sql> INSERT INTO tbl1 PARTITION (part0=0) SELECT 0;
spark-sql> INSERT INTO tbl1 PARTITION (part0=1) SELECT 1;
spark-sql> CACHE TABLE tbl1;
spark-sql> SELECT * FROM tbl1;
0	0
1	1
spark-sql> ALTER TABLE tbl1 DROP PARTITION (part0=0);
spark-sql> SELECT * FROM tbl1;
0	0
1	1

       

      Attachments

        Issue Links

          blocks

          Sub-task - The sub-task of the issue SPARK-33941 ALTER TABLE .. DROP PARTITION doesn't invalidate the cache

          • Major - Major loss of function.
          • In Progress
          is cloned by

          Sub-task - The sub-task of the issue SPARK-34011 ALTER TABLE .. RENAME TO PARTITION doesn't refresh cache

          • Major - Major loss of function.
          • Resolved
          links to

          Pull request #30983 [Github] Pull Request #30983 (MaxGekk)

          Pull request #31006 [Github] Pull Request #31006 (MaxGekk)

          Activity

            People

              maxgekk Max Gekk
              maxgekk Max Gekk
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: