Details
-
Documentation
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
ghx-label-10
Description
The documentation of the query option 'RUNTIME_FILTER_WAIT_TIME_MS' at https://github.com/apache/impala/blob/master/docs/topics/impala_runtime_filter_wait_time_ms.xml#L37-L43 as provided in the following describes the meaning of this query option.
The <codeph>RUNTIME_FILTER_WAIT_TIME_MS</codeph> query option
adjusts the settings for the runtime filtering feature.
It specifies a time in milliseconds that each scan node waits for
runtime filters to be produced by other plan fragments.
However the description above is not entirely accurate in that the wait time is with respect to the time when a runtime filter was registered (within QueryState::InitFilterBank()) instead of the time when a scan node is calling ScanNode::WaitForRuntimeFilters(). For instance if a scan node started so late that when ScanNode::WaitForRuntimeFilters() was called, the amount of time passed since the registration of this runtime filter was already greater than the value of 'RUNTIME_FILTER_WAIT_TIME_MS', this scan node would not be waiting for the runtime filter. Refer to https://github.com/apache/impala/blob/master/be/src/runtime/runtime-filter.cc#L86-L87 for further details.
bool RuntimeFilter::WaitForArrival(int32_t timeout_ms) const { unique_lock<mutex> l(arrival_mutex_); while (arrival_time_.Load() == 0) { int64_t ms_since_registration = MonotonicMillis() - registration_time_; int64_t ms_remaining = timeout_ms - ms_since_registration; if (ms_remaining <= 0) break; if (injection_delay_ > 0) SleepForMs(injection_delay_); arrival_cv_.WaitFor(l, ms_remaining * MICROS_PER_MILLI); } return arrival_time_.Load() != 0; }
We should revise the documentation to make it a bit clearer.