Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-5699

Support special character for bitmap measure column & computed column

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 5.0-beta
    • 5.0.0
    • Modeling
    • None

    Description

      Root Cause
      The Spark org.apache.spark.sql.functions.expr function cannot recognize special characters such as Chinese characters and needs to be enclosed in backticks.
      Dev Design
      Identify the places where expr is called and add compatibility handling for special characters:
      Build the computable column 

      FlatTableAndDictBase#concatCCs

      Build dictionary 

      DFBuilderHelper#selectColumnsInTable, DFBuilderHelper#chooseSuitableCols

      Build dictionary 

      DFDictionaryBuilder#wrapCol

       


       

      Root Cause

      Spark org.apache.spark.sql.functions.expr 函数不能识别中文等特殊字符,需要用反引号包裹。

      Dev Design

      找出 expr 调用处,增加对特殊字符的兼容处理:

      1. 构建可计算列 FlatTableAndDictBase#concatCCs
      1. 构建字典 DFBuilderHelper#selectColumnsInTableDFBuilderHelper#chooseSuitableCols
      1. 构建字典 DFDictionaryBuilder#wrapCol

      {{}}
       

      Attachments

        Activity

          People

            ygjia Yaguang Jia
            ygjia Yaguang Jia
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: