Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-6528

Planner setting the wrong number of records to read (Parquet Reader)

    XMLWordPrintableJSON

Details

    Description

      • Recently fixed the Flat Parquet reader to honor the number of records to read
      • Though few tests failed:
        TestUnionDistinct.testUnionDistinctEmptySides:356 Different number of records returned expected:<5> but was:<1>
        TestUnionAll.testUnionAllEmptySides:355 Different number of records returned expected:<5> but was:<1>
      • I debugged one of them and realized the Planner was setting the wrong number of rows to read (in this case, one)
      • You can put a break point and see this happening:
        Class: ParquetGroupScan
        Method: updateRowGroupInfo(long maxRecords)

      Attachments

        Issue Links

          is caused by

          Improvement - An improvement or enhancement to an existing feature or task. DRILL-5846 Improve Parquet Reader Performance for Flat Data types

          • Major - Major loss of function.
          • Resolved
          is part of

          Improvement - An improvement or enhancement to an existing feature or task. DRILL-4517 Reading emtpy Parquet file failes with java.lang.IllegalArgumentException

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              arina Arina Ielchiieva
              sachouche Salim Achouche
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: