Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-23775

Flaky test: DataFrameRangeSuite

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.4.0
    • 2.3.1, 2.4.0
    • SQL, Tests
    • None

    Description

      DataFrameRangeSuite.test("Cancelling stage in a query with Range.") stays sometimes in an infinite loop and times out the build.

      I presume the original intention of this test is to start a job with range and just cancel it.
      The submitted job has 2 stages but I think the author tried to cancel the first stage with ID 0 which is not the case here:

      eventually(timeout(10.seconds), interval(1.millis)) {
        assert(DataFrameRangeSuite.stageToKill > 0)
      }
      

      All in all if the first stage is slower than 10 seconds it throws TestFailedDueToTimeoutException and cancelStage will be never ever called.

      Attachments

        1. filtered.log
          914 kB
          Gabor Somogyi
        2. filtered_more_logs.log
          911 kB
          Gabor Somogyi

        Activity

          People

            gsomogyi Gabor Somogyi
            gsomogyi Gabor Somogyi
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: