Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Incomplete
-
2.4.0
-
None
Description
Currently, grouped aggregate and window aggregate uses different EvalType, however, they map to the same user facing type PandasUDFType.GROUPED_MAP.
It makes sense to have one user facing type because it (PandasUDFType.GROUPED_MAP) can be used in both groupby and window operation.
However, the mismatching between PandasUDFType and EvalType can be confusing to developers. We should clarify and/or improve this.
See discussion at: https://github.com/apache/spark/pull/22620#discussion_r222452544