|
|
|
SPARK-39607
|
SPARK-22386
DataSourceV2: Distribution and ordering support V2 function in writing
|
Cheng Pan
|
Cheng Pan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-37262
|
SPARK-22386
Not log empty aggregate and group by in JDBCScan
|
Huaxin Gao
|
Huaxin Gao
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-37220
|
SPARK-22386
Do not split input file for Parquet reader with aggregate push down
|
Cheng Su
|
Cheng Su
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-37167
|
SPARK-22386
Add benchmark for aggregate push down
|
Unassigned
|
Cheng Su
|
|
Open |
Unresolved
|
|
|
|
|
|
|
|
SPARK-36647
|
SPARK-22386
Push down filter by partition column for Aggregate (Min/Max/Count) for Parquet
|
Huaxin Gao
|
Huaxin Gao
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-36646
|
SPARK-22386
Push down group by partition column for Aggregate (Min/Max/Count) for Parquet
|
Huaxin Gao
|
Huaxin Gao
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-36645
|
SPARK-22386
Aggregate (Min/Max/Count) push down for Parquet
|
Huaxin Gao
|
Huaxin Gao
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-34960
|
SPARK-22386
Aggregate (Min/Max/Count) push down for ORC
|
Cheng Su
|
Cheng Su
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-34952
|
SPARK-22386
DS V2 Aggregate push down
|
Huaxin Gao
|
Huaxin Gao
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-34230
|
SPARK-22386
Let AQE determine the right parallelism in DistributionAndOrderingUtils
|
Unassigned
|
Anton Okolnychyi
|
|
Open |
Unresolved
|
|
|
|
|
|
|
|
SPARK-34183
|
SPARK-22386
DataSource V2: Support required distribution and ordering in SS
|
Anton Okolnychyi
|
Anton Okolnychyi
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-34049
|
SPARK-22386
DataSource V2: Use Write abstraction in StreamExecution
|
Anton Okolnychyi
|
Anton Okolnychyi
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-34026
|
SPARK-22386
DataSource V2: Inject repartition and sort nodes to satisfy required distribution and ordering
|
Anton Okolnychyi
|
Anton Okolnychyi
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-33808
|
SPARK-22386
DataSource V2: Build logical writes in the optimizer
|
Anton Okolnychyi
|
Anton Okolnychyi
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-33807
|
SPARK-22386
Data Source V2: Remove read specific distributions
|
Unassigned
|
Anton Okolnychyi
|
|
Open |
Unresolved
|
|
|
|
|
|
|
|
SPARK-33779
|
SPARK-22386
DataSource V2: API to request distribution and ordering on write
|
Anton Okolnychyi
|
Anton Okolnychyi
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-29248
|
SPARK-22386
Pass in number of partitions to BuildWriter
|
Ximo Guanter
|
Ximo Guanter
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-28612
|
SPARK-22386
DataSourceV2: Add new DataFrameWriter API for v2
|
Ryan Blue
|
Ryan Blue
|
|
Resolved |
Done
|
|
|
|
|
|
|
|
SPARK-28555
|
SPARK-22386
Recover options and properties and pass them back into the v1 API
|
Unassigned
|
Xin Ren
|
|
Open |
Unresolved
|
|
|
|
|
|
|
|
SPARK-25700
|
SPARK-22386
Avoid to create a readsupport at write path in Data Source V2
|
Hyukjin Kwon
|
Hyukjin Kwon
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-25460
|
SPARK-22386
DataSourceV2: Structured Streaming does not respect SessionConfigSupport
|
Hyukjin Kwon
|
Hyukjin Kwon
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-25280
|
SPARK-22386
Add support for USING syntax for DataSourceV2
|
Unassigned
|
Hyukjin Kwon
|
|
Resolved |
Incomplete
|
|
|
|
|
|
|
|
SPARK-25127
|
SPARK-22386
DataSourceV2: Remove SupportsPushDownCatalystFilters
|
Reynold Xin
|
Ryan Blue
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-24991
|
SPARK-22386
use InternalRow in DataSourceWriter
|
Wenchen Fan
|
Wenchen Fan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-24990
|
SPARK-22386
merge ReadSupport and ReadSupportWithSchema
|
Wenchen Fan
|
Wenchen Fan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-24971
|
SPARK-22386
remove SupportsDeprecatedScanRow
|
Wenchen Fan
|
Wenchen Fan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-24478
|
SPARK-22386
DataSourceV2 should push filters and projection at physical plan conversion
|
Ryan Blue
|
Ryan Blue
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-24130
|
SPARK-22386
Data Source V2: Join Push Down
|
Unassigned
|
Jia Li
|
|
Resolved |
Incomplete
|
|
|
|
|
|
|
|
SPARK-24073
|
SPARK-22386
DataSourceV2: Rename DataReaderFactory to InputPartition.
|
Ryan Blue
|
Ryan Blue
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-23889
|
SPARK-22386
DataSourceV2: Add interfaces to pass required sorting and clustering for writes
|
Unassigned
|
Ryan Blue
|
|
Resolved |
Implemented
|
|
|
|
|
|
|
|
SPARK-23418
|
SPARK-22386
DataSourceV2 should not allow userSpecifiedSchema without ReadSupportWithSchema
|
Ryan Blue
|
Ryan Blue
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-23398
|
SPARK-22386
DataSourceV2 should provide a way to get a source's schema.
|
Unassigned
|
Ryan Blue
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-23341
|
SPARK-22386
DataSourceOptions should handle path and table names to avoid confusion.
|
Wenchen Fan
|
Ryan Blue
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-23325
|
SPARK-22386
DataSourceV2 readers should always produce InternalRow.
|
Ryan Blue
|
Ryan Blue
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-23323
|
SPARK-22386
DataSourceV2 should use the output commit coordinator.
|
Ryan Blue
|
Ryan Blue
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-23321
|
SPARK-22386
DataSourceV2 should apply some validation when writing.
|
Unassigned
|
Ryan Blue
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-23268
|
SPARK-22386
Reorganize packages in data source V2
|
Gengliang Wang
|
Gengliang Wang
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-23204
|
SPARK-22386
DataSourceV2 should support named tables in DataFrameReader, DataFrameWriter
|
Unassigned
|
Ryan Blue
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-23203
|
SPARK-22386
DataSourceV2 should use immutable trees.
|
Ryan Blue
|
Ryan Blue
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-22391
|
SPARK-22386
add `MetadataCreationSupport` trait to separate data and metadata handling at write path
|
Unassigned
|
Wenchen Fan
|
|
Resolved |
Incomplete
|
|
|
|
|
|
|
|
SPARK-22390
|
SPARK-22386
Aggregate push down
|
Unassigned
|
Wenchen Fan
|
|
Resolved |
Duplicate
|
|
|
|
|
|
|
|
SPARK-22388
|
SPARK-22386
Limit push down
|
Unassigned
|
Wenchen Fan
|
|
Resolved |
Incomplete
|
|
|
|
|