Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3537

DefaultContainerExecutor has a race condn. with multiple concurrent containers

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 0.23.0
    • 0.23.1
    • None
    • None

    Description

      DCE relies cwd before calling ContainerLocalizer.runLocalization. However, with multiple containers setting cwd on same localFS reference leads to race.

      Attachments

        1. MAPREDUCE-3537.patch
          1 kB
          Arun Murthy

        Issue Links

          Activity

            acmurthy Arun Murthy added a comment -

            Quick fix to add synchronization to unblock Pig, Oozie etc.

            We should redo the DCE more throughly.

            acmurthy Arun Murthy added a comment - Quick fix to add synchronization to unblock Pig, Oozie etc. We should redo the DCE more throughly.
            hadoopqa Hadoop QA added a comment -

            -1 overall. Here are the results of testing the latest attachment
            http://issues.apache.org/jira/secure/attachment/12507110/MAPREDUCE-3537.patch
            against trunk revision .

            +1 @author. The patch does not contain any @author tags.

            -1 tests included. The patch doesn't appear to include any new or modified tests.
            Please justify why no new tests are needed for this patch.
            Also please list what manual steps were performed to verify this patch.

            +1 javadoc. The javadoc tool did not generate any warning messages.

            +1 javac. The applied patch does not increase the total number of javac compiler warnings.

            +1 eclipse:eclipse. The patch built with eclipse:eclipse.

            +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

            +1 release audit. The applied patch does not increase the total number of release audit warnings.

            +1 core tests. The patch passed unit tests in .

            +1 contrib tests. The patch passed contrib unit tests.

            Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1429//testReport/
            Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1429//console

            This message is automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12507110/MAPREDUCE-3537.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1429//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1429//console This message is automatically generated.
            mahadev Mahadev Konar added a comment -

            +1 the patch looks good.

            mahadev Mahadev Konar added a comment - +1 the patch looks good.
            acmurthy Arun Murthy added a comment -

            I just committed this.

            acmurthy Arun Murthy added a comment - I just committed this.
            hudson Hudson added a comment -

            Integrated in Hadoop-Hdfs-0.23-Commit #264 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/264/)
            Merge -c 1213575 from trunk to branch-0.23 to fix MAPREDUCE-3537.

            acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1213576
            Files :

            • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
            • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
            hudson Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #264 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/264/ ) Merge -c 1213575 from trunk to branch-0.23 to fix MAPREDUCE-3537 . acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1213576 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
            hudson Hudson added a comment -

            Integrated in Hadoop-Hdfs-trunk-Commit #1480 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1480/)
            MAPREDUCE-3537. Fix race condition in DefaultContainerExecutor which led to container localization occuring in wrong directories.

            acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1213575
            Files :

            • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
            • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
            hudson Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1480 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1480/ ) MAPREDUCE-3537 . Fix race condition in DefaultContainerExecutor which led to container localization occuring in wrong directories. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1213575 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
            hudson Hudson added a comment -

            Integrated in Hadoop-Common-trunk-Commit #1406 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1406/)
            MAPREDUCE-3537. Fix race condition in DefaultContainerExecutor which led to container localization occuring in wrong directories.

            acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1213575
            Files :

            • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
            • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
            hudson Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1406 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1406/ ) MAPREDUCE-3537 . Fix race condition in DefaultContainerExecutor which led to container localization occuring in wrong directories. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1213575 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
            hudson Hudson added a comment -

            Integrated in Hadoop-Common-0.23-Commit #274 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/274/)
            Merge -c 1213575 from trunk to branch-0.23 to fix MAPREDUCE-3537.

            acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1213576
            Files :

            • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
            • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
            hudson Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #274 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/274/ ) Merge -c 1213575 from trunk to branch-0.23 to fix MAPREDUCE-3537 . acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1213576 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
            hudson Hudson added a comment -

            Integrated in Hadoop-Mapreduce-trunk-Commit #1430 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1430/)
            MAPREDUCE-3537. Fix race condition in DefaultContainerExecutor which led to container localization occuring in wrong directories.

            acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1213575
            Files :

            • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
            • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
            hudson Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1430 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1430/ ) MAPREDUCE-3537 . Fix race condition in DefaultContainerExecutor which led to container localization occuring in wrong directories. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1213575 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
            hudson Hudson added a comment -

            Integrated in Hadoop-Mapreduce-0.23-Commit #286 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/286/)
            Merge -c 1213575 from trunk to branch-0.23 to fix MAPREDUCE-3537.

            acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1213576
            Files :

            • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
            • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
            hudson Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #286 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/286/ ) Merge -c 1213575 from trunk to branch-0.23 to fix MAPREDUCE-3537 . acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1213576 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
            hudson Hudson added a comment -

            Integrated in Hadoop-Hdfs-0.23-Build #106 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/106/)
            Merge -c 1213575 from trunk to branch-0.23 to fix MAPREDUCE-3537.

            acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1213576
            Files :

            • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
            • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
            hudson Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #106 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/106/ ) Merge -c 1213575 from trunk to branch-0.23 to fix MAPREDUCE-3537 . acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1213576 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
            hudson Hudson added a comment -

            Integrated in Hadoop-Mapreduce-0.23-Build #124 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/124/)
            Merge -c 1213575 from trunk to branch-0.23 to fix MAPREDUCE-3537.

            acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1213576
            Files :

            • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
            • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
            hudson Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #124 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/124/ ) Merge -c 1213575 from trunk to branch-0.23 to fix MAPREDUCE-3537 . acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1213576 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
            hudson Hudson added a comment -

            Integrated in Hadoop-Hdfs-trunk #893 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/893/)
            MAPREDUCE-3537. Fix race condition in DefaultContainerExecutor which led to container localization occuring in wrong directories.

            acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1213575
            Files :

            • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
            • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
            hudson Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #893 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/893/ ) MAPREDUCE-3537 . Fix race condition in DefaultContainerExecutor which led to container localization occuring in wrong directories. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1213575 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
            hudson Hudson added a comment -

            Integrated in Hadoop-Mapreduce-trunk #926 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/926/)
            MAPREDUCE-3537. Fix race condition in DefaultContainerExecutor which led to container localization occuring in wrong directories.

            acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1213575
            Files :

            • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
            • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
            hudson Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #926 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/926/ ) MAPREDUCE-3537 . Fix race condition in DefaultContainerExecutor which led to container localization occuring in wrong directories. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1213575 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java

            People

              acmurthy Arun Murthy
              acmurthy Arun Murthy
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: