Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3652

Fix resource transfer in subplans with limits

    XMLWordPrintableJSON

Details

    Description

      There is a tricky corner case in our resource transfer model with subplans and limits. The problem is that the limit in the subplan may mean that the exec node is reset before it has returned its full output. The resource transfer logic generally attaches resources to batches at specific points in the output, e.g. end of partition, end of block, so it's possible that batches returned before the Reset() may reference resources that have not yet been transferred. It's unclear if we test this scenario consistently or if it's always handled correctly.

      One example is this query, reported in IMPALA-5456:

      select c_custkey, c_mktsegment, o_orderkey, o_orderdate
from customer c,
  (select o1.o_orderkey, o2.o_orderdate
   from c.c_orders o1, c.c_orders o2
   where o1.o_orderkey = o2.o_orderkey limit 10) v limit 500;

      Attachments

        Issue Links

          blocks

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-5456 impala crashes in impala::Tuple::DeepCopyVarlenData trying to memcpy a null pointer

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-5456 impala crashes in impala::Tuple::DeepCopyVarlenData trying to memcpy a null pointer

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-5456 impala crashes in impala::Tuple::DeepCopyVarlenData trying to memcpy a null pointer

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-4179 Remove RowBatch::MarkNeedsDeepCopy() memory management API

          • Major - Major loss of function.
          • Open

          Activity

            People

              twmarshall Thomas Tauber-Marshall
              tarmstrong Tim Armstrong
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: