Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.1.0
    • 3.1.0
    • SQL
    • None

    Description

      Spark SQL support some window function like NTH_VALUE
      If we specify window frame like

      UNBOUNDED PRECEDING AND CURRENT ROW
      

      or

      UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING
      

      We can elimate some calculations.
      For example: if we execute the SQL show below:

      SELECT NTH_VALUE(col,
               2) OVER(ORDER BY rank UNBOUNDED PRECEDING
              AND CURRENT ROW)
      FROM tab;
      

      The output for row number greater than 1, return the fixed value. otherwise, return null. So we just calculate the value once and notice whether the row number less than 2.
      UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING is simpler.

      Attachments

        Activity

          apachespark Apache Spark added a comment -

          User 'beliefer' has created a pull request for this issue:
          https://github.com/apache/spark/pull/29800

          apachespark Apache Spark added a comment - User 'beliefer' has created a pull request for this issue: https://github.com/apache/spark/pull/29800
          apachespark Apache Spark added a comment -

          User 'beliefer' has created a pull request for this issue:
          https://github.com/apache/spark/pull/29800

          apachespark Apache Spark added a comment - User 'beliefer' has created a pull request for this issue: https://github.com/apache/spark/pull/29800
          cloud_fan Wenchen Fan added a comment -

          Issue resolved by pull request 29800
          https://github.com/apache/spark/pull/29800

          cloud_fan Wenchen Fan added a comment - Issue resolved by pull request 29800 https://github.com/apache/spark/pull/29800
          apachespark Apache Spark added a comment -

          User 'cloud-fan' has created a pull request for this issue:
          https://github.com/apache/spark/pull/30261

          apachespark Apache Spark added a comment - User 'cloud-fan' has created a pull request for this issue: https://github.com/apache/spark/pull/30261

          People

            beliefer Jiaan Geng
            beliefer Jiaan Geng
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: