Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-5441

SerDeUtil Pair RDD to python conversion doesn't accept empty RDDs

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.1.1, 1.2.0
    • 1.2.2, 1.3.0
    • PySpark

    Description

      SerDeUtil.pairRDDToPython and SerDeUtil.pythonToPairRDD rely on rdd.first() which throws an exception if the RDD is empty. We should be able to handle the empty RDD case because this doesn't prevent a valid RDD from being created.

      Attachments

        Activity

          People

            mnazario Michael Nazario
            mnazario Michael Nazario
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: