Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7361

Clean shared state pollution to avoid flaky tests in class TestTaskProgressReporter

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • None
    • None
    • test

    Description

      What is the purpose of this change

      This PR is to clean the polluted shared status among the 3 tests: 

      Test1: org.apache.hadoop.mapred.TestTaskProgressReporter.testBytesWrittenRespectingLimit
      Test2: org.apache.hadoop.mapred.TestTaskProgressReporter.testScratchDirSize
      Test3: org.apache.hadoop.mapred.TestTaskProgressReporter.testTaskProgress

      Test1 and Test2 pollute the shared status with Test3, which can make test3 fail. It may be better to clean state pollutions so that some other tests won't fail in the future due to the shared state polluted by this test.

      Reproduce the test failures

      Run the tests in the same JVM in the following orders:

      • Test1 and Test3
      • Test2 and Test3

      Expected result

      The tests should run successfully when multiple tests that use this shared state are run in the same JVM.

      Actual result

      • Run Test1 and Test3, Test3 fails:
       [ERROR] TestTaskProgressReporter.testTaskProgress:267 expected:<[2]> but was:<[3]>
      • Run Test2 and Test3, Test3 fails:
       [ERROR] TestTaskProgressReporter.testTaskProgress:267 expected:<[2]> but was:<[11]>

      Fix

      Clean the value of statusUpdateTimes to 0 at the start of Test3 to clean the pollution from Test1 and Test2.

      Link to PR: https://github.com/apache/hadoop/pull/3298

      Attachments

        1. MAPREDUCE-7361.001.patch
          1.0 kB
          Yang Chen

        Issue Links

          Activity

            People

              Unassigned Unassigned
              LALAYANG Yang Chen
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 10m
                  10m