Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-2953

Chained sortPartition() calls produce incorrect results in Scala DataSet API

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 0.9, 0.10.0, 1.0.0
    • 0.9.2, 0.10.0
    • API / DataSet, API / Scala
    • None

    Description

      Chaining for sortPartition() calls does not work correctly in Scala DataSet API.

      val x: DataSet[(Int, Int, Int)] = ... //
      x
        .sortPartition(0, Order.DESCENDING)
        .sortPartition(2, Order.ASCENDING)
        .print()
      
      // result
      (1,2,3)
      (1,3,4)
      (1,2,5)
      (3,7,8)
      
      // should be
      (3,7,8)
      (1,2,3)
      (1,3,4)
      (1,2,5)
      

      The same program in the Java DataSet API works correctly.

      Attachments

        Activity

          People

            Unassigned Unassigned
            fhueske Fabian Hueske
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: