Details
-
Bug
-
Status: In Progress
-
Major
-
Resolution: Unresolved
-
Impala 4.5.0
-
None
-
None
-
ghx-label-9
Description
If a runtime filter impacts the results of a fragment, then the tuple cache key needs to incorporate information about the generation of that runtime filter. This needs to include information about the base tables that impact the runtime filter.
For example, suppose there is a join. The build side of the join produces a runtime filter that gets delivered to the probe side of the join. The tuple cache key for the probe side of the join will need to include a representation of the runtime filter. If the table on the build side of the join changes, the tuple cache key for the probe side needs to change due to the possible difference in the runtime filter.
This can also impact eligibility. In theory, the build side of a join could be constructed from a source with a limit specified, and this can result in non-determinism. Since the build of the runtime filter is not deterministic, the consumer of the runtime filter is not deterministic and can't participate in tuple caching.