Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0
    • 3.1.0
    • SQL
    • None

    Description

      Spark SQL support these bracketed comments:
      Case 1:

      /* This is an example of SQL which should not execute:
       * select 'multi-line';
       */
      

      Case 2:

      /*
      SELECT 'trailing' as x1; -- inside block comment
      */
      

      But Spark SQL not support nested bracketed comments show below:

      Case 3:

      /* This block comment surrounds a query which itself has a block comment...
      SELECT /* embedded single line */ 'embedded' AS x2;
      */
      

      Case 4:

      SELECT -- continued after the following block comments...
      /* Deeply nested comment.
         This includes a single apostrophe to make sure we aren't decoding this part as a string.
      SELECT 'deep nest' AS n1;
      /* Second level of nesting...
      SELECT 'deeper nest' as n2;
      /* Third level of nesting...
      SELECT 'deepest nest' as n3;
      */
      Hoo boy. Still two deep...
      */
      Now just one deep...
      */
      'deeply nested example' AS sixth;
      

      bracketed comments
      Bracketed comments are introduced by /* and end with */. 

      https://www.ibm.com/support/knowledgecenter/en/SSCJDQ/com.ibm.swg.im.dashdb.sql.ref.doc/doc/c0056402.html

      https://www.postgresql.org/docs/11/sql-syntax-lexical.html#SQL-SYNTAX-COMMENTS
      Feature ID:  T351

      Attachments

        Activity

          beliefer Jiaan Geng added a comment -

          I added test cases in AnalysisErrorSuite show below:

           

            errorTest(
              "test comments",
              CatalystSqlParser.parsePlan("-- single comment\nSELECT hex(DISTINCT a) FROM TaBlE"),
              "DISTINCT or FILTER specified, but hex is not an aggregate function" :: Nil)
           
            errorTest(
              "multi comments",
              CatalystSqlParser.parsePlan("/* multi comments\n */SELECT hex(DISTINCT a) FROM TaBlE"),
              "DISTINCT or FILTER specified, but hex is not an aggregate function" :: Nil)
          

           

          and added test cases in PlanParserSuite show below:

           

              assertOverlayPlans(
                "-- single comment\nSELECT OVERLAY('Spark SQL' PLACING '_' FROM 6)",
                new Overlay(Literal("Spark SQL"), Literal("_"), Literal(6))
              )
           
              assertOverlayPlans(
                """/* This is an example of SQL which should not execute:\n
                  | * select 'multi-line';\n
                  | */SELECT OVERLAY('Spark SQL' PLACING '_' FROM 6)""".stripMargin,
                new Overlay(Literal("Spark SQL"), Literal("_"), Literal(6))
              )
          

           

          All the test cases passed.

          All the test cases can't passed If I remove code from SqlBase.g4 show below:

           

          -SIMPLE_COMMENT
          -    : '--' ~[\r\n]* '\r'? '\n'? -> channel(HIDDEN)
          -    ;
          -
          -BRACKETED_EMPTY_COMMENT
          -    : '/**/' -> channel(HIDDEN)
          -    ;
          -
          -BRACKETED_COMMENT
          -    : '/*' ~[+] .*? '*/' -> channel(HIDDEN)
          -    ;
          

           

           

           

           

          beliefer Jiaan Geng added a comment - I added test cases in AnalysisErrorSuite show below:     errorTest(      "test comments" ,     CatalystSqlParser.parsePlan( "-- single comment\nSELECT hex(DISTINCT a) FROM TaBlE" ),      "DISTINCT or FILTER specified, but hex is not an aggregate function" :: Nil)     errorTest(      "multi comments" ,     CatalystSqlParser.parsePlan( " /* multi comments\n */ SELECT hex(DISTINCT a) FROM TaBlE" ),      "DISTINCT or FILTER specified, but hex is not an aggregate function" :: Nil)   and added test cases in PlanParserSuite show below:       assertOverlayPlans(        "-- single comment\nSELECT OVERLAY( 'Spark SQL' PLACING '_' FROM 6)" ,        new Overlay(Literal( "Spark SQL" ), Literal( "_" ), Literal(6))     )       assertOverlayPlans(       """/* This is an example of SQL which should not execute:\n         | * select 'multi-line' ;\n         | */SELECT OVERLAY( 'Spark SQL' PLACING '_' FROM 6)""".stripMargin,        new Overlay(Literal( "Spark SQL" ), Literal( "_" ), Literal(6))     )   All the test cases passed. All the test cases can't passed If I remove code from SqlBase.g4 show below:   -SIMPLE_COMMENT -    : '--' ~[\r\n]* '\r' ? '\n' ? -> channel(HIDDEN) -    ; - -BRACKETED_EMPTY_COMMENT -    : ' /**/ ' -> channel(HIDDEN) -    ; - -BRACKETED_COMMENT -    : ' /*' ~[+] .*? '*/ ' -> channel(HIDDEN) -    ;        
          smilegator Xiao Li added a comment - https://github.com/apache/spark/blob/master/sql/core/src/test/resources/sql-tests/inputs/postgreSQL/comments.sql You can try to enable these tests
          Gengliang.Wang Gengliang Wang added a comment - This issue is resolved in https://github.com/apache/spark/pull/27495

          People

            beliefer Jiaan Geng
            yumwang Yuming Wang
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: