Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-35098

Incorrect results for queries like "10 >= y" on tables using Filesystem connector and Orc format

    XMLWordPrintableJSON

Details

    Description

      When working with ORC files, there is an issue with evaluation of SQL queries containing expressions with a literal as the first operand. Specifically, the query 10 >= y does not always return the correct result.

      This test added to OrcFileSystemITCase.java fails on the second check:

       

      @TestTemplate
      void testOrcFilterPushDownLiteralFirst() throws ExecutionException, InterruptedException {
          super.tableEnv()
                  .executeSql("insert into orcLimitTable values('a', 10, 10)")
                  .await();
      
          List<Row> expected = Collections.singletonList(Row.of(10));
          check("select y from orcLimitTable where y <= 10", expected);
          check("select y from orcLimitTable where 10 >= y", expected);
      }
      
      Results do not match for query:
        select y from orcLimitTable where 10 >= y
      Results
       == Correct Result - 1 ==   == Actual Result - 0 ==
      !+I[10]    

      The checks are equivalent and should evaluate to the same result. But the second query doesn't return the record with y=10.

      The table is defined as:

      create table orcLimitTable (
          x string,
          y int,
          a int) 
      with (
          'connector' = 'filesystem',
          'path' = '/tmp/junit4374176500101507155/junit7109291529844202275/',
          'format'='orc')

       

      Attachments

        Activity

          People

            empathy87 Andrey Gaskov
            empathy87 Andrey Gaskov
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: