Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-4541

Make sure query planner does not generate operators with mixed convention trait

    XMLWordPrintableJSON

Details

    • Bug
    • Status: In Progress
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None

    Description

      Per the discussion [1] in the PR of DRILL-4531, we should fix the query planner rules used in Drill planning, such that it will not generate Rels with mixed convention trait. For instance, a LogicalFilter should only have child with NONE convention; it should not have child with LOGICAL convention.

      The mixed Rels will cause planner either hang (as reported in DRILL-4531 and DRILL-3257), or do wasted work by firing rules against the mixed Rels.

      I think the reason that we have such mixed rels is we have different kinds of rules, used in a single Volcano planning phase.
      1) Rule matchs base class Filter/Project, etc only.
      2) Rule matches LogicalFilter/LogicalProject, etc
      3) Rule matches DrillFilter/DrillProject, etc.
      3) Rule uses copy() method to generate a new Rel
      4) Rule uses RelFactory to generate a new Rel.
      5) convent rule, which convert from Calcite logical (NONE/Enumerable) to Drill logical (LOGICAL)

      For instance, ProjectMergeRule, which matches base Project, yet uses default RelFactory, will match both LogicalProject and DrillProject, but produce LogicalProject as outcome. That will cause the mixed rels.

      2 things we may consider to fix this:
      1) Separate the convent rules from the other transformation rules. Apply convert rule first, then all the transformation rules match DrillLogical only.
      2) Every rule that Drill uses, except for convert rules, should assert the convention of input and output have the same convention.

      [1] https://github.com/apache/drill/pull/444

      Attachments

        Activity

          People

            jni Jinfeng Ni
            jni Jinfeng Ni
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: