Description
Root Cause
The problem lies in the method discardInvalidColsAndMeasForBrokenModel. The expression of cc is not expanded, while innerExpression is expanded. This method only collects the original columns in the table. When there is nested computed column, some computed columns will not be collected, resulting in this method mistakenly judging certain cc columns as invalid.
Fix Design
When parsing cc, still use innerExpression, but innerExpression is compatible with Spark syntax. Directly handing it over to Calcite for SQL parsing and obtaining tables and columns may encounter syntax issues such as unsupported functions. Therefore, simply call the QueryUtil.adaptCalciteSyntax method.
Root Cause
问题出在discardInvalidColsAndMeasForBrokenModel这个方法, cc 的 expression 是不展开的,而 innerExpression 是展开的。这个方法中只收集了表中的原始列,当存在可计算列的嵌套是,那么有些可计算列是不会被收集的,从而这个方法就误判了某些 cc 列是非法的。
Fix Design
在解析 cc 的时候依然使用 innerExpression,但 innerExpression 是 spark 语法兼容的,直接交给 Calcite 执行 SQL 解析获取表和列可能会存在函数不支持等语法问题,因此调用一下 QueryUtil.adaptCalciteSyntax 这个方法即可。