Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-2204

DISTINCT statement over UNION ALL subquery asserts during execution with streaming aggregation

    XMLWordPrintableJSON

Details

    Description

      0: jdbc:drill:schema=dfs> select distinct sq.x1, sq.x2, sq.x3 from ( select a1, b1, c1 from t1 union all select a2, b2, c2 from t2 ) as sq(x1,x2,x3);
      +------------+------------+------------+
      |     x1     |     x2     |     x3     |
      +------------+------------+------------+
      Query failed: RemoteRpcException: Failure while running fragment., Failure while reading vector.  Expected vector class of org.apache.drill.exec.vector.NullableVarCharVector but was holding vector class org.apache.drill.exec.vector.NullableIntVector. [ dd2fedd7-bbee-40a4-8a26-9ca86fa774f6 on atsqa4-134.qa.lab:31010 ]
      [ dd2fedd7-bbee-40a4-8a26-9ca86fa774f6 on atsqa4-134.qa.lab:31010 ]
      
      
      java.lang.RuntimeException: java.sql.SQLException: Failure while executing query.
              at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
              at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
              at sqlline.SqlLine.print(SqlLine.java:1809)
              at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
              at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
              at sqlline.SqlLine.dispatch(SqlLine.java:889)
              at sqlline.SqlLine.begin(SqlLine.java:763)
              at sqlline.SqlLine.start(SqlLine.java:498)
              at sqlline.SqlLine.main(SqlLine.java:460)
      

      Plan:

      00-01      Project(x1=[$0], x2=[$1], x3=[$2])
      00-02        StreamAgg(group=[{0, 1, 2}])
      00-03          Sort(sort0=[$0], sort1=[$1], sort2=[$2], dir0=[ASC], dir1=[ASC], dir2=[ASC])
      00-04            Project(x1=[$0], x2=[$1], x3=[$2])
      00-05              UnionAll(all=[true])
      00-07                Project(a1=[$2], b1=[$1], c1=[$0])
      00-09                  Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=maprfs:/aggregation/sanity/t1]], selectionRoot=/aggregation/sanity/t1, numFiles=1, columns=[`a1`, `b1`, `c1`]]])
      00-06                Project(a2=[$1], b2=[$0], c2=[$2])
      00-08                  Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=maprfs:/aggregation/sanity/t2]], selectionRoot=/aggregation/sanity/t2, numFiles=1, columns=[`a2`, `b2`, `c2`]]])
      

      Query works if columns in the union query have the same name.

      0: jdbc:drill:schema=dfs> select distinct sq.x1, sq.x2, sq.x3 from ( select a1, b1, c1 from t1 union all select a1, b1, c1 from t4 ) as sq(x1,x2,x3);
      +------------+------------+------------+
      |     x1     |     x2     |     x3     |
      +------------+------------+------------+
      | 1          | aaaaa      | 2015-01-01 |
      | 2          | bbbbb      | 2015-01-02 |
      | 3          | ccccc      | 2015-01-03 |
      | 4          | null       | 2015-01-04 |
      | 5          | eeeee      | 2015-01-05 |
      | 6          | fffff      | 2015-01-06 |
      | 7          | ggggg      | 2015-01-07 |
      | 9          | iiiii      | null       |
      | 10         | jjjjj      | 2015-01-10 |
      | null       | hhhhh      | 2015-01-08 |
      +------------+------------+------------+
      10 rows selected (0.131 seconds)
      

      It's possible, that bug is caused by drill-2203, but I'm filing it anyway, because the way it fails it is different and will need to be verified after it is fixed. Tables for the query are attached in drill-2203

      from drillbit.log

      2015-02-10 23:10:10,387 [2b25702c-e73f-3af8-810b-cf7555854ab5:frag:0:0] WARN  o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path `a1`, returning null instance.
      2015-02-10 23:10:10,387 [2b25702c-e73f-3af8-810b-cf7555854ab5:frag:0:0] WARN  o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path `b1`, returning null instance.
      2015-02-10 23:10:10,387 [2b25702c-e73f-3af8-810b-cf7555854ab5:frag:0:0] WARN  o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path `c1`, returning null instance.
      2015-02-10 23:10:10,389 [2b25702c-e73f-3af8-810b-cf7555854ab5:frag:0:0] WARN  o.a.d.e.w.fragment.FragmentExecutor - Error while initializing or executing fragment
      java.lang.IllegalStateException: Failure while reading vector.  Expected vector class of org.apache.drill.exec.vector.NullableVarCharVector but was holding vector class org.apache.drill.exec.vector.NullableIntVector.
              at org.apache.drill.exec.record.VectorContainer.getValueAccessorById(VectorContainer.java:239) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
              at org.apache.drill.exec.record.AbstractRecordBatch.getValueAccessorById(AbstractRecordBatch.java:191) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
              at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.getValueAccessorById(IteratorValidatorBatchIterator.java:110) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
              at org.apache.drill.exec.test.generated.SingleBatchSorterGen6510.doSetup(SingleBatchSorterTemplate.java:211) ~[na:na]
              at org.apache.drill.exec.test.generated.SingleBatchSorterGen6510.setup(SingleBatchSorterTemplate.java:42) ~[na:na]
              at org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.innerNext(ExternalSortBatch.java:315) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
              at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
              at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
              at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:99) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
              at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:89) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
              at org.apache.drill.exec.physical.impl.aggregate.StreamingAggBatch.innerNext(StreamingAggBatch.java:105) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
              at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
              at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
              at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:99) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
              at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:89) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
              at org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
              at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:134) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
              at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
              at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
              at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:67) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
              at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:97) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
              at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:57) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
              at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:116) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
              at org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:303) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_71]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_71]
              at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
      2015-02-10 23:10:10,390 [2b25702c-e73f-3af8-810b-cf7555854ab5:frag:0:0] ERROR o.a.drill.exec.ops.FragmentContext - Fragment Context received failure.
      

      Attachments

        Issue Links

          Activity

            People

              seanhychu Sean Hsuan-Yi Chu
              vicky Victoria Markman
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: