Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-11346

Migrated partitioned Iceberg tables might return ERROR when WHERE condition is used on partition column

    XMLWordPrintableJSON

Details

    • ghx-label-12

    Description

      [localhost:21050] default> select * from functional_parquet.iceberg_alltypes_part where p_bool=false;
      Fetched 0 row(s) in 0.11s
      
      [localhost:21050] default> select * from functional_parquet.iceberg_alltypes_part where p_bool=true;
      ERROR: Unable to find SchemaNode for path 'functional_parquet.iceberg_alltypes_part.p_bool' in the schema of file 'hdfs://localhost:20500/test-warehouse/iceberg_test/hadoop_catalog/ice/iceberg_alltypes_part/p_bool=true/p_int=1/p_bigint=11/p_float=1.1/p_double=2.222/p_decimal=123.321/p_date=2022-02-22/p_string=impala/000000_0'.
      
      [localhost:21050] default> select * from functional_parquet.iceberg_alltypes_part where i=3;
      Fetched 0 row(s) in 0.12s
      
      [localhost:21050] default> select * from functional_parquet.iceberg_alltypes_part where i=1;
      +---+--------+-------+----------+---------------+----------+-----------+------------+----------+
      | i | p_bool | p_int | p_bigint | p_float       | p_double | p_decimal | p_date     | p_string |
      +---+--------+-------+----------+---------------+----------+-----------+------------+----------+
      | 1 | true   | 1     | 11       | 1.10000002384 | 2.222    | 123.321   | 2022-02-22 | impala   |
      +---+--------+-------+----------+---------------+----------+-----------+------------+----------+
      Fetched 1 row(s) in 0.12s
      
      [localhost:21050] default> select * from functional_parquet.iceberg_alltypes_part where p_int=1;
      ERROR: Unable to find SchemaNode for path 'functional_parquet.iceberg_alltypes_part.p_int' in the schema of file 'hdfs://localhost:20500/test-warehouse/iceberg_test/hadoop_catalog/ice/iceberg_alltypes_part/p_bool=true/p_int=1/p_bigint=11/p_float=1.1/p_double=2.222/p_decimal=123.321/p_date=2022-02-22/p_string=impala/000000_0'.
      
      [localhost:21050] default> select * from functional_parquet.iceberg_alltypes_part where p_int=3;
      Fetched 0 row(s) in 0.11s

      So we don't get incorrect results at least, but getting errors on partition column values that are existing.

      It seems like it works well with ORC.

      Attachments

        Activity

          People

            boroknagyz Zoltán Borók-Nagy
            boroknagyz Zoltán Borók-Nagy
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: