Bulk Operation

SPARK-44101 Upgrade Pandas to 2.1.2

SPARK-44101 Remove remaining deprecated Pandas APIs from Spark 3.4.0

SPARK-44101 Enable `GroupbySplitApplyTests.test_split_apply_combine_on_series` for pandas 2.0.0.

SPARK-44101 Change the default value for `numeric_only`.

SPARK-44101 Upgrade Pandas to 2.1.1

SPARK-44101 Remove boolean inputs for inclusive from Series.between

SPARK-44101 Remove `col_space` parameter from `DataFrame.to_latex`

SPARK-44101 Remove `inplace` parameter from `Categorical` APIs

SPARK-44101 Remove deprecated Index APIs

SPARK-44101 Support Pandas 2.1.0

SPARK-44101 Update document related to removed Index object

SPARK-44101 Support stat functions

SPARK-44101 Support value_counts for pandas 2.0.0.

SPARK-44101 Enable GroupbyAggregateTests.test_aggregate for pandas 2.0.0.

SPARK-44101 Match behavior with pandas for `SeriesStringTests.test_string_replace`

SPARK-44101 Warning for the pandas-related behavior changes in next major release

SPARK-44101 Enable CategoricalTests for pandas 2.0.0.

SPARK-44101 Enable GroupByTests for pandas 2.0.0.

SPARK-44101 Enable DataFrameSlowTests for pandas 2.0.0.

SPARK-44101 Enable DataFramePlotMatplotlibTests for pandas 2.0.0.

SPARK-44101 Enable SeriesDateTimeTests for pandas 2.0.0.

SPARK-44101 Enable SeriesTests for pandas 2.0.0.

SPARK-44101 Enable GroupBySlowTests for pandas 2.0.0.

SPARK-44101 Enable CategoricalTests.test_groupby_apply_without_shortcut for pandas 2.0.0.

SPARK-44101 Enable DataFrameTests.test_all for pandas 2.0.0.

SPARK-44101 Enable DataFrameTests.test_reindex for pandas 2.0.0.

SPARK-44101 Enable DataFrameSlowTests.test_quantile for pandas 2.0.0.

SPARK-44101 Enable DataFrameSlowTests.test_cov for pandas 2.0.0.

SPARK-44101 Enable SummarizerTests.test_summarize_dataframe for pandas 2.0.0.

SPARK-44101 Enable FeatureTests.test_max_abs_scaler for pandas 2.0.0.

SPARK-44101 Enable FeatureTests.test_standard_scaler for pandas 2.0.0.

SPARK-44101 Enable SeriesDateTimeTests.test_year for pandas 2.0.0.

SPARK-44101 Enable SeriesDateTimeTests.test_weekday for pandas 2.0.0.

SPARK-44101 Enable SeriesDateTimeTests.test_second for pandas 2.0.0.

SPARK-44101 Enable SeriesDateTimeTests.test_quarter for pandas 2.0.0.

SPARK-44101 Enable SeriesDateTimeTests.test_month for pandas 2.0.0.

SPARK-44101 Enable SeriesDateTimeTests.test_minute for pandas 2.0.0.

SPARK-44101 Enable SeriesDateTimeTests.test_microsecond for pandas 2.0.0.

SPARK-44101 Enable SeriesDateTimeTests.test_hour for pandas 2.0.0.

SPARK-44101 Enable SeriesDateTimeTests.test_daysinmonth for pandas 2.0.0.

SPARK-44101 Enable SeriesDateTimeTests.test_days_in_month for pandas 2.0.0.

SPARK-44101 Enable SeriesDateTimeTests.test_dayofyear for pandas 2.0.0.

SPARK-44101 Enable SeriesDateTimeTests.test_dayofweek for pandas 2.0.0.

SPARK-44101 Enable SeriesDateTimeTests.test_day for pandas 2.0.0.

SPARK-44101 Enable DataFramePlotMatplotlibTests.test_kde_plot for pandas 2.0.0.

SPARK-44101 Enable DataFramePlotMatplotlibTests.test_hist_plot for pandas 2.0.0.

SPARK-44101 Enable NamespaceTests.test_date_range for pandas 2.0.0.

SPARK-44101 Enable GroupByTests.test_apply for pandas 2.0.0.

SPARK-44101 Enable GroupByTests.test_mean for pandas 2.0.0.

SPARK-44101 Enable GroupByTests.test_apply_without_shortcut for pandas 2.0.0.

SPARK-44101 Enable TimedeltaIndexTests.test_properties for pandas 2.0.0.

SPARK-44101 Enable DatetimeIndexTests.test_indexer_between_time for pandas 2.0.0.

SPARK-44101 Enable DataFramePlotMatplotlibTests.test_scatter_plot for pandas 2.0.0.

SPARK-44101 Enable DataFramePlotMatplotlibTests.test_pie_plot for pandas 2.0.0.

SPARK-44101 Enable DataFramePlotMatplotlibTests.test_line_plot for pandas 2.0.0.

SPARK-44101 Enable DataFramePlotMatplotlibTests.test_barh_plot for pandas 2.0.0.

SPARK-44101 Enable DataFramePlotMatplotlibTests.test_barh_plot_with_x_y for pandas 2.0.0.

SPARK-44101 Enable DataFramePlotMatplotlibTests.test_bar_with_x_y for pandas 2.0.0.

SPARK-44101 Enable DataFramePlotMatplotlibTests.test_bar_plot for pandas 2.0.0.

SPARK-44101 Enable DataFramePlotMatplotlibTests.test_area_plot_y for pandas 2.0.0.

SPARK-44101 Enable DataFramePlotMatplotlibTests.test_area_plot_stacked_false for pandas 2.0.0.

SPARK-44101 Enable DataFramePlotMatplotlibTests.test_area_plot for pandas 2.0.0.

SPARK-44101 Enable CategoricalIndexTests.test_remove_categories for pandas 2.0.0.

SPARK-44101 Enable IndexesTests.test_union for pandas 2.0.0.

SPARK-44101 Enable IndexesTests for pandas 2.0.0.

SPARK-44101 Enable IndexesTests.test_index_basic for pandas 2.0.0.

SPARK-44101 Enable CategoricalTests.test_remove_categories for pandas 2.0.0.

SPARK-44101 Enable DateOpsTests.test_sub for pandas 2.0.0.

SPARK-44101 Enable DateOpsTests.test_rsub for pandas 2.0.0.

SPARK-44101 Enable CategoricalIndexTests.test_categories_setter for pandas 2.0.0.

SPARK-44101 Enable CategoricalIndexTests.test_factorize for pandas 2.0.0.

SPARK-44101 Enable CategoricalTests.test_categories_setter for pandas 2.0.0.

SPARK-44101 Enable CategoricalTests.test_as_ordered_unordered for pandas 2.0.0.

SPARK-44101 Enable CategoricalTests.test_factorize for pandas 2.0.0.

SPARK-44101 Enable CsvTests.test_read_csv_with_squeeze for pandas 2.0.0.

SPARK-44101 Enable DataFrameTests.test_append for pandas 2.0.0.

SPARK-44101 Enable DataFrameConversionTests.test_to_latex for pandas 2.0.0.

SPARK-44101 Enable DataFrameSlowTests.test_mad for pandas 2.0.0.

SPARK-44101 Enable DataFrameSlowTests.test_iteritems for pandas 2.0.0.

SPARK-44101 Enable DataFrameSlowTests.test_product for pandas 2.0.0.

SPARK-44101 Enable DataFrameSlowTests.test_between_time for pandas 2.0.0.

SPARK-44101 Enable DataFrameSlowTests.test_describe for pandas 2.0.0.

SPARK-44101 Enable GroupByTests.test_groupby_multiindex_columns for pandas 2.0.0.

SPARK-44101 Enable GroupByTests.test_basic_stat_funcs for pandas 2.0.0.

SPARK-44101 Enable GroupByTests.test_mad for pandas 2.0.0.

SPARK-44101 Enable GroupByTests.test_nth for pandas 2.0.0.

SPARK-44101 Enable GroupByTests.test_prod for pandas 2.0.0.

SPARK-44101 Enable SeriesTests.test_factorize for pandas 2.0.0.

SPARK-44101 Enable MultiIndex test for IndexesTests.test_difference

SPARK-44101 Enable ArrowTests.test_toPandas_empty_columns for pandas 2.0.0.

SPARK-44101 Enable StatsTests.test_stat_functions_with_no_numeric_columns for pandas 2.0.0.

SPARK-44101 Enable StatsTests.test_axis_on_dataframe for pandas 2.0.0.

SPARK-44101 Enable StatsTests.test_cov_corr_meta for pandas 2.0.0.

SPARK-44101 Enable SeriesTests.test_product for pandas 2.0.0.

SPARK-44101 Enable SeriesTests.test_iteritems for pandas 2.0.0.

SPARK-44101 Enable SeriesTests.test_between_time for pandas 2.0.0.

SPARK-44101 Enable SeriesStringTests.test_string_split for pandas 2.0.0.

SPARK-44101 Enable SeriesStringTests.test_string_rsplit for pandas 2.0.0.

SPARK-44101 Enable SeriesStringTests.test_string_replace for pandas 2.0.0.

SPARK-44101 Enable SeriesTests.test_quantile for pandas 2.0.0.

SPARK-44101 Enable SeriesTests.test_mad for pandas 2.0.0.

SPARK-44101 Enable SeriesTests.test_between for pandas 2.0.0.

SPARK-44101 Enable SeriesTests.test_astype for pandas 2.0.0.

SPARK-44101 Enable SeriesTests.test_append for pandas 2.0.0.

SPARK-44101 Enable SeriesTests.test_value_counts for pandas 2.0.0.

SPARK-44101 Enable SeriesTests.test_rank for pandas 2.0.0.

SPARK-44101 Enable SeriesDateTimeTests.test_date_subtraction for pandas 2.0.0.

SPARK-44101 Enable OpsOnDiffFramesGroupByTests.test_groupby_different_lengths for pandas 2.0.0.

SPARK-44101 Enable OpsOnDiffFramesGroupByTests for pandas 2.0.0.

SPARK-44101 Enable SeriesConversionTests.test_to_latex for pandas 2.0.0.

SPARK-44101 Ignore the names of MultiIndex when axis=1 for concat

SPARK-44101 Enable RollingTests.test_groupby_rolling_count for pandas 2.0.0.

SPARK-44101 Enable RollingTests.test_rolling_count for pandas 2.0.0.

SPARK-44101 Enable GroupBySlowTests.test_split_apply_combine_on_series for pandas 2.0.0.

SPARK-44101 Enable GroupBySlowTests.test_value_counts for pandas 2.0.0.

SPARK-44101 Match `GroupBy.nth` behavior with new pandas behavior

SPARK-44101 Enable test_to_latex by supporting jinja2>=3.0.0

SPARK-44101 Make DataFrameGroupBy.sum support for string type columns

SPARK-44101 Generate proper warning on different behavior with numeric_only

SPARK-44101 Investigate DataFrame.sort_values with pandas behavior.

SPARK-44101 Match behavior with DataFrame.reindex with specifying `index`.

SPARK-44101 Fix DatetimeIndex.microsecond to return 'int32' instead of 'int64' type of Index.

SPARK-44101 MultiIndex.append not checking names for equality

SPARK-44101 PySpark 3.4.0 cannot convert timestamp-typed objects to pandas with pandas 2.0

SPARK-44101 Upgrade pandas to 2.0.0

SPARK-44101 Add migration notes for update to supported pandas version.

SPARK-44101 Add `inclusive` parameter for date_range

SPARK-44101 Add `inclusive` parameter for (DataFrame|Series).between_time

SPARK-44101 Add `show_counts` parameter for DataFrame.info

SPARK-44101 Warning for the pandas-related behavior changes in next major release

SPARK-44101 Support `isocalendar`

SPARK-44101 Deprecate & remove the APIs that will be removed in pandas 2.0.