Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-23775

Flaky test: DataFrameRangeSuite

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.4.0
    • 2.3.1, 2.4.0
    • SQL, Tests
    • None

    Description

      DataFrameRangeSuite.test("Cancelling stage in a query with Range.") stays sometimes in an infinite loop and times out the build.

      I presume the original intention of this test is to start a job with range and just cancel it.
      The submitted job has 2 stages but I think the author tried to cancel the first stage with ID 0 which is not the case here:

      eventually(timeout(10.seconds), interval(1.millis)) {
        assert(DataFrameRangeSuite.stageToKill > 0)
      }
      

      All in all if the first stage is slower than 10 seconds it throws TestFailedDueToTimeoutException and cancelStage will be never ever called.

      Attachments

        1. filtered_more_logs.log
          911 kB
          Gabor Somogyi
        2. filtered.log
          914 kB
          Gabor Somogyi

        Activity

          People

            gsomogyi Gabor Somogyi
            gsomogyi Gabor Somogyi
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: