Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-35337 pandas API on Spark: Separate basic operations into data type based structures
  3. SPARK-35599

Introduce a way to compare series of array for older pandas

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.2.0
    • 3.2.0
    • PySpark
    • None

    Description

      PySpark tests ComplexOpsTest.test_add failed with older pandas e.g. v1.0.1, with the ValueError The truth value of an array with more than one element is ambiguous. Use a.any() or a.all().

      We need to introduce a way to check the equality when the data are arrays.

      Attachments

        Activity

          People

            XinrongM Xinrong Meng
            XinrongM Xinrong Meng
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: