Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7359

Clean shared state pollution to avoid flaky tests in testCombiner

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • None
    • None
    • test
    • None

    Description

      What is the purpose of this change

      This PR is to fix 2 non-idempotent flaky tests:

      org.apache.hadoop.mapred.TestOldCombinerGrouping.testCombiner org.apache.hadoop.mapreduce.TestNewCombinerGrouping.testCombiner

      Why the tests failed

      For test testcombiner, it writes to the directory TEST_ROOT_DIR whose path is determined by UUID. When running this test twice, the TEST_ROOT_DIR created by the first running already exists, so the second running cannot create the directory with the same name, which will lead to a RuntimeException. This will pollute the shared state. It may be better to clean state pollutions so that some other tests won't fail in the future due to the shared state polluted by this test.

      Reproduce test failure

      Run the test twice in the same JVM.

      Expected result

      The tests should run successfully without any failure when multiple tests that use this state are run in the same JVM.

      Actual result

      Failure from test run:

      java.lang.RuntimeException: Could not create test dir: /home/...

      Fix

      Fully delete the directory created for this test when it ends.

       

      Link to PR:

      Attachments

        Activity

          People

            Unassigned Unassigned
            LALAYANG Yang Chen
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: