Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-17615

Getting "java.lang.RuntimeException: Distinct columns cannot exist in Aggregate "

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Duplicate
    • 2.0.0
    • None
    • SQL
    • None

    Description

      I execute:

      select platform, 
              collect_set(user_auth) as paid_types,
              count(distinct sessionid) as sessions
          from non_hss.session
          where
              event = 'stop' and platform != 'testplatform' and
              not (month = MONTH(current_date()) AND year = YEAR(current_date()) and day = day(current_date())) and
              (
                  (month >= MONTH(add_months(CURRENT_DATE(), -5)) AND year = YEAR(add_months(CURRENT_DATE(), -5)))
                  OR
                  (month <= MONTH(add_months(CURRENT_DATE(), -5)) AND year > YEAR(add_months(CURRENT_DATE(), -5)))
              )
          group by platform
      

      I get:

      java.lang.RuntimeException: Distinct columns cannot exist in Aggregate operator containing aggregate functions which don't support partial aggregation.
      

      IT WORKED IN 1.6.2. I've read error 5 times, and read code once. I still don't understand what I do incorrectly.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              epahomov Egor Pahomov
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: