Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-12014

Spark SQL query containing semicolon is broken in Beeline (related to HIVE-11100)

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Incomplete
    • 1.5.2
    • None
    • SQL

    Description

      Actually it is known hive issue: https://issues.apache.org/jira/browse/HIVE-11100

      patch available: https://reviews.apache.org/r/35907/diff/1

      but Spark uses its own maven dependencies for hive (org.spark-project.hive), we can not use this patch to fix the problem, it would be better if you can fix this in spark's hive package.

      In spark's beeline, the error message will be:

      0: jdbc:hive2://host:10000/> CREATE TABLE beeline_tb (c1 int, c2 string) ROW FORMAT DELIMITED FIELDS TERMINATED BY ';' LINES TERMINATED BY '\n';
Error: org.apache.spark.sql.AnalysisException: mismatched input '<EOF>' expecting StringLiteral near 'BY' in table row format's field separator; line 1 pos 87 (state=,code=0)

0: jdbc:hive2://host:10000/> CREATE TABLE beeline_tb (c1 int, c2 string) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\;' LINES TERMINATED BY '\n';
Error: org.apache.spark.sql.AnalysisException: mismatched input '<EOF>' expecting StringLiteral near 'BY' in table row format's field separator; line 1 pos 88 (state=,code=0)

0: jdbc:hive2://host:10000/> SELECT str_to_map(other_data,';','=')['key_name'] FROM some_logs WHERE log_date = '20151125' limit 5;
Error: org.apache.spark.sql.AnalysisException: cannot recognize input near '<EOF>' '<EOF>' '<EOF>' in select expression; line 1 pos 30 (state=,code=0)

0: jdbc:hive2://host:10000/> SELECT str_to_map(other_data,'\;','=')['key_name'] FROM some_logs WHERE log_date = '20151125' limit 5;
Error: org.apache.spark.sql.AnalysisException: cannot recognize input near '<EOF>' '<EOF>' '<EOF>' in select expression; line 1 pos 31 (state=,code=0)

      Attachments

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. SPARK-12497 thriftServer does not support semicolon in sql

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-11100 Beeline should escape semi-colon in queries

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Umbrella - An overarching type made of sub-tasks SPARK-23710 Upgrade the built-in Hive to 2.3.5 for hadoop-3.2

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              chutium Teng Qiu
              Votes:
              4 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: