Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6202

Configuration item Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY is disregarded

Details

    • Reviewed

    Description

      Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY (yarn.dispatcher.exit-on-error) always be true no matter what value in configuration files. This misleads users. Two solutions:

      1. Remove the configuration item and provide a method to allow exitOnDispatchException/shouldExitOnError to be false to enable related unit tests. There is no need for false value in a real daemon since daemons should crash if its dispatcher quit.
      2. Make it default true instead of false, so that we don't need to hard code it to be true in RM and NM, it is still configurable, and also provide method to enable related unit tests.

      Other than that, the code around it needs to refactor. public static final for a variable of interface isn't necessary, and YARN related configure item should be in class YarnConfiguration.

      Attachments

        1. YARN-6202.001.patch
          16 kB
          Yufei Gu
        2. YARN-6202.002.patch
          20 kB
          Yufei Gu
        3. YARN-6202.003.patch
          21 kB
          Yufei Gu
        4. YARN-6202.004.patch
          22 kB
          Yufei Gu

        Issue Links

          Activity

            yufeigu Yufei Gu added a comment -

            Option 1 sounds good to me. Any thoughts? vinodkv

            yufeigu Yufei Gu added a comment - Option 1 sounds good to me. Any thoughts? vinodkv
            yufeigu Yufei Gu added a comment -

            Uploaded patch v1.
            Remove configuration items: Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY, and add methods for AsyncDispatcher and EventDispatcher to disable exit on exception/error, which are invoked by unit tests.

            yufeigu Yufei Gu added a comment - Uploaded patch v1. Remove configuration items: Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY, and add methods for AsyncDispatcher and EventDispatcher to disable exit on exception/error, which are invoked by unit tests.
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 13s Docker mode activated.
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
            0 mvndep 0m 33s Maven dependency ordering for branch
            +1 mvninstall 15m 46s trunk passed
            +1 compile 19m 2s trunk passed
            +1 checkstyle 2m 13s trunk passed
            +1 mvnsite 3m 14s trunk passed
            +1 mvneclipse 1m 52s trunk passed
            +1 findbugs 5m 9s trunk passed
            +1 javadoc 2m 31s trunk passed
            0 mvndep 0m 19s Maven dependency ordering for patch
            +1 mvninstall 2m 33s the patch passed
            +1 compile 13m 6s the patch passed
            +1 javac 13m 6s the patch passed
            -0 checkstyle 2m 1s root: The patch generated 2 new + 313 unchanged - 5 fixed = 315 total (was 318)
            +1 mvnsite 3m 1s the patch passed
            +1 mvneclipse 1m 50s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 findbugs 6m 44s the patch passed
            +1 javadoc 2m 28s the patch passed
            +1 unit 2m 37s hadoop-yarn-common in the patch passed.
            -1 unit 0m 39s hadoop-yarn-server-nodemanager in the patch failed.
            -1 unit 0m 45s hadoop-yarn-server-resourcemanager in the patch failed.
            +1 unit 9m 6s hadoop-mapreduce-client-app in the patch passed.
            +1 unit 3m 16s hadoop-mapreduce-client-hs in the patch passed.
            +1 asflicense 0m 41s The patch does not generate ASF License warnings.
            124m 28s



            Subsystem Report/Notes
            Docker Image:yetus/hadoop:a9ad5d6
            JIRA Issue YARN-6202
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12861109/YARN-6202.001.patch
            Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
            uname Linux 5886c5f6133c 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
            git revision trunk / 4966a6e
            Default Java 1.8.0_121
            findbugs v3.0.0
            checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/15426/artifact/patchprocess/diff-checkstyle-root.txt
            unit https://builds.apache.org/job/PreCommit-YARN-Build/15426/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
            unit https://builds.apache.org/job/PreCommit-YARN-Build/15426/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
            Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15426/testReport/
            modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: .
            Console output https://builds.apache.org/job/PreCommit-YARN-Build/15426/console
            Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files. 0 mvndep 0m 33s Maven dependency ordering for branch +1 mvninstall 15m 46s trunk passed +1 compile 19m 2s trunk passed +1 checkstyle 2m 13s trunk passed +1 mvnsite 3m 14s trunk passed +1 mvneclipse 1m 52s trunk passed +1 findbugs 5m 9s trunk passed +1 javadoc 2m 31s trunk passed 0 mvndep 0m 19s Maven dependency ordering for patch +1 mvninstall 2m 33s the patch passed +1 compile 13m 6s the patch passed +1 javac 13m 6s the patch passed -0 checkstyle 2m 1s root: The patch generated 2 new + 313 unchanged - 5 fixed = 315 total (was 318) +1 mvnsite 3m 1s the patch passed +1 mvneclipse 1m 50s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 6m 44s the patch passed +1 javadoc 2m 28s the patch passed +1 unit 2m 37s hadoop-yarn-common in the patch passed. -1 unit 0m 39s hadoop-yarn-server-nodemanager in the patch failed. -1 unit 0m 45s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 9m 6s hadoop-mapreduce-client-app in the patch passed. +1 unit 3m 16s hadoop-mapreduce-client-hs in the patch passed. +1 asflicense 0m 41s The patch does not generate ASF License warnings. 124m 28s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6202 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12861109/YARN-6202.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 5886c5f6133c 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4966a6e Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/15426/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/15426/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/15426/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15426/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/15426/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
            yufeigu Yufei Gu added a comment -

            The compiler errors are weird. I don't have them locally. My guess is the Hadoop QA partially compile the project.

            yufeigu Yufei Gu added a comment - The compiler errors are weird. I don't have them locally. My guess is the Hadoop QA partially compile the project.
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 17s Docker mode activated.
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
            0 mvndep 0m 16s Maven dependency ordering for branch
            +1 mvninstall 14m 19s trunk passed
            +1 compile 17m 19s trunk passed
            +1 checkstyle 2m 9s trunk passed
            +1 mvnsite 3m 8s trunk passed
            +1 mvneclipse 1m 50s trunk passed
            +1 findbugs 5m 54s trunk passed
            +1 javadoc 2m 38s trunk passed
            0 mvndep 0m 18s Maven dependency ordering for patch
            +1 mvninstall 2m 55s the patch passed
            +1 compile 18m 12s the patch passed
            +1 javac 18m 12s the patch passed
            -0 checkstyle 2m 13s root: The patch generated 2 new + 313 unchanged - 5 fixed = 315 total (was 318)
            +1 mvnsite 3m 19s the patch passed
            +1 mvneclipse 2m 9s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 findbugs 5m 49s the patch passed
            +1 javadoc 2m 49s the patch passed
            +1 unit 2m 38s hadoop-yarn-common in the patch passed.
            -1 unit 12m 29s hadoop-yarn-server-nodemanager in the patch failed.
            -1 unit 40m 27s hadoop-yarn-server-resourcemanager in the patch failed.
            -1 unit 10m 19s hadoop-mapreduce-client-app in the patch failed.
            +1 unit 4m 16s hadoop-mapreduce-client-hs in the patch passed.
            +1 asflicense 0m 51s The patch does not generate ASF License warnings.
            182m 27s



            Reason Tests
            Failed junit tests hadoop.yarn.server.resourcemanager.TestRMHA
              hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
            Timed out junit tests org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.TestResourceLocalizationService
              org.apache.hadoop.yarn.server.nodemanager.containermanager.TestContainerManagerRecovery
              org.apache.hadoop.yarn.server.nodemanager.containermanager.container.TestContainer
              org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.TestRMContainerImpl
              org.apache.hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore
              org.apache.hadoop.yarn.server.resourcemanager.recovery.TestFSRMStateStore
              org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore
              org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl



            Subsystem Report/Notes
            Docker Image:yetus/hadoop:a9ad5d6
            JIRA Issue YARN-6202
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12861109/YARN-6202.001.patch
            Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
            uname Linux 19015b68a570 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
            git revision trunk / 7297e82
            Default Java 1.8.0_121
            findbugs v3.0.0
            checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/15441/artifact/patchprocess/diff-checkstyle-root.txt
            unit https://builds.apache.org/job/PreCommit-YARN-Build/15441/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
            unit https://builds.apache.org/job/PreCommit-YARN-Build/15441/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
            unit https://builds.apache.org/job/PreCommit-YARN-Build/15441/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
            Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15441/testReport/
            modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: .
            Console output https://builds.apache.org/job/PreCommit-YARN-Build/15441/console
            Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files. 0 mvndep 0m 16s Maven dependency ordering for branch +1 mvninstall 14m 19s trunk passed +1 compile 17m 19s trunk passed +1 checkstyle 2m 9s trunk passed +1 mvnsite 3m 8s trunk passed +1 mvneclipse 1m 50s trunk passed +1 findbugs 5m 54s trunk passed +1 javadoc 2m 38s trunk passed 0 mvndep 0m 18s Maven dependency ordering for patch +1 mvninstall 2m 55s the patch passed +1 compile 18m 12s the patch passed +1 javac 18m 12s the patch passed -0 checkstyle 2m 13s root: The patch generated 2 new + 313 unchanged - 5 fixed = 315 total (was 318) +1 mvnsite 3m 19s the patch passed +1 mvneclipse 2m 9s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 49s the patch passed +1 javadoc 2m 49s the patch passed +1 unit 2m 38s hadoop-yarn-common in the patch passed. -1 unit 12m 29s hadoop-yarn-server-nodemanager in the patch failed. -1 unit 40m 27s hadoop-yarn-server-resourcemanager in the patch failed. -1 unit 10m 19s hadoop-mapreduce-client-app in the patch failed. +1 unit 4m 16s hadoop-mapreduce-client-hs in the patch passed. +1 asflicense 0m 51s The patch does not generate ASF License warnings. 182m 27s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestRMHA   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer Timed out junit tests org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.TestResourceLocalizationService   org.apache.hadoop.yarn.server.nodemanager.containermanager.TestContainerManagerRecovery   org.apache.hadoop.yarn.server.nodemanager.containermanager.container.TestContainer   org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.TestRMContainerImpl   org.apache.hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore   org.apache.hadoop.yarn.server.resourcemanager.recovery.TestFSRMStateStore   org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore   org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6202 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12861109/YARN-6202.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 19015b68a570 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 7297e82 Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/15441/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/15441/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/15441/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/15441/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15441/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/15441/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
            yufeigu Yufei Gu added a comment -

            Uploaded patch v2 to fix test failures.

            yufeigu Yufei Gu added a comment - Uploaded patch v2 to fix test failures.
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 18s Docker mode activated.
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 9 new or modified test files.
            0 mvndep 0m 16s Maven dependency ordering for branch
            +1 mvninstall 13m 41s trunk passed
            +1 compile 16m 5s trunk passed
            +1 checkstyle 2m 0s trunk passed
            +1 mvnsite 2m 47s trunk passed
            +1 mvneclipse 1m 42s trunk passed
            +1 findbugs 4m 46s trunk passed
            +1 javadoc 2m 14s trunk passed
            0 mvndep 0m 17s Maven dependency ordering for patch
            +1 mvninstall 2m 10s the patch passed
            +1 compile 15m 40s the patch passed
            +1 javac 15m 40s the patch passed
            -0 checkstyle 2m 1s root: The patch generated 4 new + 359 unchanged - 5 fixed = 363 total (was 364)
            +1 mvnsite 3m 2s the patch passed
            +1 mvneclipse 1m 57s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 findbugs 5m 59s the patch passed
            +1 javadoc 2m 37s the patch passed
            +1 unit 2m 43s hadoop-yarn-common in the patch passed.
            +1 unit 13m 21s hadoop-yarn-server-nodemanager in the patch passed.
            -1 unit 38m 40s hadoop-yarn-server-resourcemanager in the patch failed.
            +1 unit 8m 54s hadoop-mapreduce-client-app in the patch passed.
            +1 unit 3m 25s hadoop-mapreduce-client-hs in the patch passed.
            +1 asflicense 0m 42s The patch does not generate ASF License warnings.
            170m 34s



            Reason Tests
            Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart
            Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore



            Subsystem Report/Notes
            Docker Image:yetus/hadoop:a9ad5d6
            JIRA Issue YARN-6202
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12861341/YARN-6202.002.patch
            Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
            uname Linux 50e9ed2c623a 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
            git revision trunk / b58777a
            Default Java 1.8.0_121
            findbugs v3.0.0
            checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/15447/artifact/patchprocess/diff-checkstyle-root.txt
            unit https://builds.apache.org/job/PreCommit-YARN-Build/15447/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
            Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15447/testReport/
            modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: .
            Console output https://builds.apache.org/job/PreCommit-YARN-Build/15447/console
            Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 9 new or modified test files. 0 mvndep 0m 16s Maven dependency ordering for branch +1 mvninstall 13m 41s trunk passed +1 compile 16m 5s trunk passed +1 checkstyle 2m 0s trunk passed +1 mvnsite 2m 47s trunk passed +1 mvneclipse 1m 42s trunk passed +1 findbugs 4m 46s trunk passed +1 javadoc 2m 14s trunk passed 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 2m 10s the patch passed +1 compile 15m 40s the patch passed +1 javac 15m 40s the patch passed -0 checkstyle 2m 1s root: The patch generated 4 new + 359 unchanged - 5 fixed = 363 total (was 364) +1 mvnsite 3m 2s the patch passed +1 mvneclipse 1m 57s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 59s the patch passed +1 javadoc 2m 37s the patch passed +1 unit 2m 43s hadoop-yarn-common in the patch passed. +1 unit 13m 21s hadoop-yarn-server-nodemanager in the patch passed. -1 unit 38m 40s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 8m 54s hadoop-mapreduce-client-app in the patch passed. +1 unit 3m 25s hadoop-mapreduce-client-hs in the patch passed. +1 asflicense 0m 42s The patch does not generate ASF License warnings. 170m 34s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6202 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12861341/YARN-6202.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 50e9ed2c623a 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b58777a Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/15447/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/15447/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15447/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/15447/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 17s Docker mode activated.
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 9 new or modified test files.
            0 mvndep 0m 14s Maven dependency ordering for branch
            +1 mvninstall 14m 6s trunk passed
            +1 compile 16m 56s trunk passed
            +1 checkstyle 2m 4s trunk passed
            +1 mvnsite 2m 56s trunk passed
            +1 mvneclipse 1m 43s trunk passed
            +1 findbugs 5m 3s trunk passed
            +1 javadoc 2m 17s trunk passed
            0 mvndep 0m 16s Maven dependency ordering for patch
            +1 mvninstall 2m 14s the patch passed
            +1 compile 14m 45s the patch passed
            +1 javac 14m 45s the patch passed
            -0 checkstyle 2m 2s root: The patch generated 5 new + 359 unchanged - 5 fixed = 364 total (was 364)
            +1 mvnsite 3m 5s the patch passed
            +1 mvneclipse 2m 3s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 findbugs 6m 17s the patch passed
            +1 javadoc 2m 33s the patch passed
            +1 unit 2m 30s hadoop-yarn-common in the patch passed.
            +1 unit 13m 10s hadoop-yarn-server-nodemanager in the patch passed.
            -1 unit 38m 40s hadoop-yarn-server-resourcemanager in the patch failed.
            +1 unit 8m 44s hadoop-mapreduce-client-app in the patch passed.
            +1 unit 3m 14s hadoop-mapreduce-client-hs in the patch passed.
            +1 asflicense 0m 40s The patch does not generate ASF License warnings.
            171m 18s



            Reason Tests
            Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore



            Subsystem Report/Notes
            Docker Image:yetus/hadoop:a9ad5d6
            JIRA Issue YARN-6202
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12861342/YARN-6202.002.patch
            Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
            uname Linux e25de5ee9382 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
            git revision trunk / b58777a
            Default Java 1.8.0_121
            findbugs v3.0.0
            checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/15448/artifact/patchprocess/diff-checkstyle-root.txt
            unit https://builds.apache.org/job/PreCommit-YARN-Build/15448/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
            Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15448/testReport/
            modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: .
            Console output https://builds.apache.org/job/PreCommit-YARN-Build/15448/console
            Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 9 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 14m 6s trunk passed +1 compile 16m 56s trunk passed +1 checkstyle 2m 4s trunk passed +1 mvnsite 2m 56s trunk passed +1 mvneclipse 1m 43s trunk passed +1 findbugs 5m 3s trunk passed +1 javadoc 2m 17s trunk passed 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 2m 14s the patch passed +1 compile 14m 45s the patch passed +1 javac 14m 45s the patch passed -0 checkstyle 2m 2s root: The patch generated 5 new + 359 unchanged - 5 fixed = 364 total (was 364) +1 mvnsite 3m 5s the patch passed +1 mvneclipse 2m 3s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 6m 17s the patch passed +1 javadoc 2m 33s the patch passed +1 unit 2m 30s hadoop-yarn-common in the patch passed. +1 unit 13m 10s hadoop-yarn-server-nodemanager in the patch passed. -1 unit 38m 40s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 8m 44s hadoop-mapreduce-client-app in the patch passed. +1 unit 3m 14s hadoop-mapreduce-client-hs in the patch passed. +1 asflicense 0m 40s The patch does not generate ASF License warnings. 171m 18s Reason Tests Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6202 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12861342/YARN-6202.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e25de5ee9382 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b58777a Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/15448/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/15448/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15448/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/15448/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
            haibochen Haibo Chen added a comment -

            Reading comments above Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY, it seems like DISPATCHER_EXIT_ON_ERROR_KEY is deliberately set to false by default (see comment at https://issues.apache.org/jira/browse/MAPREDUCE-3634?focusedCommentId=13195964&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13195964).

            YARN related configure item should be in class YarnConfiguration

            This property is not a user-facing property, only frameworks and tests use it, I think. so whether it should go into YarnConfiguration is debatable.

            haibochen Haibo Chen added a comment - Reading comments above Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY, it seems like DISPATCHER_EXIT_ON_ERROR_KEY is deliberately set to false by default (see comment at https://issues.apache.org/jira/browse/MAPREDUCE-3634?focusedCommentId=13195964&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13195964 ). YARN related configure item should be in class YarnConfiguration This property is not a user-facing property, only frameworks and tests use it, I think. so whether it should go into YarnConfiguration is debatable.
            yufeigu Yufei Gu added a comment -

            You are right. No matter what default value is, this shouldn't be a configuration item at all. I saw people actually configure it, and thought the items could work! The reality is the daemons should always be true, and some tests need it to be false no matter what configuration is.
            I think make the default to be true should be better since it should be true for daemons. The only downside is we need to explicitly set it to false in some test cases, which is mitigated by following cases:

            1. All unit tests using dispatcher in MockRM is fine since we set it to false in MockRM.
            2. All unit tests using DrainDispatcher is fine since we set it false in DrainDispatcher as well.
            yufeigu Yufei Gu added a comment - You are right. No matter what default value is, this shouldn't be a configuration item at all. I saw people actually configure it, and thought the items could work! The reality is the daemons should always be true, and some tests need it to be false no matter what configuration is. I think make the default to be true should be better since it should be true for daemons. The only downside is we need to explicitly set it to false in some test cases, which is mitigated by following cases: All unit tests using dispatcher in MockRM is fine since we set it to false in MockRM. All unit tests using DrainDispatcher is fine since we set it false in DrainDispatcher as well.
            haibochen Haibo Chen added a comment -

            AsyncDispatcher are also used in a lot of other tests besides MockRM, so it will be a lot of test changes.

            few tests are crashing without reporting a junit failure due to sys-exit and so are hiding code bugs

            Based on this comment in MAPREDUCE-3634, I still think it is better to leave it as false by default, since it is very easy to forget to set this in a new test that uses AsyncDispatcher.

            haibochen Haibo Chen added a comment - AsyncDispatcher are also used in a lot of other tests besides MockRM, so it will be a lot of test changes. few tests are crashing without reporting a junit failure due to sys-exit and so are hiding code bugs Based on this comment in MAPREDUCE-3634 , I still think it is better to leave it as false by default, since it is very easy to forget to set this in a new test that uses AsyncDispatcher.
            yufeigu Yufei Gu added a comment -

            It won't hide code bugs since the unit tests won't pass if AsyncDispatcher crashed. People who use AsyncDispatcher assume it crash on exceptions. Hiding this is more dangerous by making it false by default.

            yufeigu Yufei Gu added a comment - It won't hide code bugs since the unit tests won't pass if AsyncDispatcher crashed. People who use AsyncDispatcher assume it crash on exceptions. Hiding this is more dangerous by making it false by default.
            haibochen Haibo Chen added a comment -

            Per offline discussion with Yufei, post to YARN-2917, AsyncDispatcher thread crash will cause JVM to exit(-1), so it is no longer valid that setting DISPATCHER_EXIT_ON_ERROR_KEY to true by default will hide unit test failures. Thus, +1 on this change.

            haibochen Haibo Chen added a comment - Per offline discussion with Yufei, post to YARN-2917 , AsyncDispatcher thread crash will cause JVM to exit(-1), so it is no longer valid that setting DISPATCHER_EXIT_ON_ERROR_KEY to true by default will hide unit test failures. Thus, +1 on this change.

            Thanks for tackling this, yufeigu. I agree with the general approach. Can you clean up the unused imports in the checkstyle warning? Also, the unit tests aren't passing:

            Running org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
            Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 80.344 sec - in org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
            
            Results :
            
            Tests run: 1620, Failures: 0, Errors: 0, Skipped: 3
            
            [INFO] ------------------------------------------------------------------------
            [INFO] BUILD FAILURE
            [INFO] ------------------------------------------------------------------------
            [INFO] Total time: 38:21.624s
            [INFO] Finished at: Fri Mar 31 01:15:42 UTC 2017
            [INFO] Final Memory: 18M/250M
            [INFO] ------------------------------------------------------------------------
            [WARNING] The requested profile "parallel-tests" could not be activated because it does not exist.
            [WARNING] The requested profile "native" could not be activated because it does not exist.
            [WARNING] The requested profile "yarn-ui" could not be activated because it does not exist.
            [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on project hadoop-yarn-server-resourcemanager: ExecutionException: java.lang.RuntimeException: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
            [ERROR] Command was /bin/sh -c cd /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager && /usr/lib/jvm/java-8-oracle/jre/bin/java -Xmx2048m -XX:+HeapDumpOnOutOfMemoryError -jar /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/surefire/surefirebooter8156637588310801099.jar /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/surefire/surefire3523297314551279161tmp /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/surefire/surefire_124814350333586959477tmp
            [ERROR] -> [Help 1]
            [ERROR] 
            [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
            [ERROR] Re-run Maven using the -X switch to enable full debug logging.
            [ERROR] 
            [ERROR] For more information about the errors and possible solutions, please read the following articles:
            [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
            
            templedf Daniel Templeton added a comment - Thanks for tackling this, yufeigu . I agree with the general approach. Can you clean up the unused imports in the checkstyle warning? Also, the unit tests aren't passing: Running org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 80.344 sec - in org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart Results : Tests run: 1620, Failures: 0, Errors: 0, Skipped: 3 [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 38:21.624s [INFO] Finished at: Fri Mar 31 01:15:42 UTC 2017 [INFO] Final Memory: 18M/250M [INFO] ------------------------------------------------------------------------ [WARNING] The requested profile "parallel-tests" could not be activated because it does not exist. [WARNING] The requested profile "native" could not be activated because it does not exist. [WARNING] The requested profile "yarn-ui" could not be activated because it does not exist. [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on project hadoop-yarn-server-resourcemanager: ExecutionException: java.lang.RuntimeException: The forked VM terminated without properly saying goodbye. VM crash or System.exit called? [ERROR] Command was /bin/sh -c cd /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager && /usr/lib/jvm/java-8-oracle/jre/bin/java -Xmx2048m -XX:+HeapDumpOnOutOfMemoryError -jar /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/surefire/surefirebooter8156637588310801099.jar /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/surefire/surefire3523297314551279161tmp /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/surefire/surefire_124814350333586959477tmp [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
            yufeigu Yufei Gu added a comment -

            Thanks templedf for the review. Uploaded patch v3 to fix style issues. The unit test failure is unrelated.

            yufeigu Yufei Gu added a comment - Thanks templedf for the review. Uploaded patch v3 to fix style issues. The unit test failure is unrelated.
            yufeigu Yufei Gu added a comment -

            My bad. I test the wrong uni test, which is TestAMRestart. The failure happens in class TestZKRMStateStore. Uploaded patch v4 for that.

            yufeigu Yufei Gu added a comment - My bad. I test the wrong uni test, which is TestAMRestart. The failure happens in class TestZKRMStateStore. Uploaded patch v4 for that.
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 3m 54s Docker mode activated.
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 10 new or modified test files.
            0 mvndep 1m 46s Maven dependency ordering for branch
            +1 mvninstall 15m 36s trunk passed
            +1 compile 16m 56s trunk passed
            +1 checkstyle 2m 2s trunk passed
            +1 mvnsite 2m 52s trunk passed
            +1 mvneclipse 1m 52s trunk passed
            +1 findbugs 4m 32s trunk passed
            +1 javadoc 2m 19s trunk passed
            0 mvndep 0m 16s Maven dependency ordering for patch
            +1 mvninstall 2m 17s the patch passed
            +1 compile 14m 53s the patch passed
            +1 javac 14m 53s the patch passed
            -0 checkstyle 2m 0s root: The patch generated 2 new + 365 unchanged - 5 fixed = 367 total (was 370)
            +1 mvnsite 3m 20s the patch passed
            +1 mvneclipse 2m 8s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 findbugs 5m 31s the patch passed
            +1 javadoc 2m 34s the patch passed
            +1 unit 2m 32s hadoop-yarn-common in the patch passed.
            +1 unit 13m 5s hadoop-yarn-server-nodemanager in the patch passed.
            -1 unit 39m 59s hadoop-yarn-server-resourcemanager in the patch failed.
            +1 unit 9m 8s hadoop-mapreduce-client-app in the patch passed.
            +1 unit 3m 28s hadoop-mapreduce-client-hs in the patch passed.
            +1 asflicense 0m 41s The patch does not generate ASF License warnings.
            178m 57s



            Reason Tests
            Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart



            Subsystem Report/Notes
            Docker Image:yetus/hadoop:612578f
            JIRA Issue YARN-6202
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12863168/YARN-6202.004.patch
            Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
            uname Linux 7f87b577b9ac 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
            git revision trunk / 0cab572
            Default Java 1.8.0_121
            findbugs v3.0.0
            checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/15616/artifact/patchprocess/diff-checkstyle-root.txt
            unit https://builds.apache.org/job/PreCommit-YARN-Build/15616/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
            Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15616/testReport/
            modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: .
            Console output https://builds.apache.org/job/PreCommit-YARN-Build/15616/console
            Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 3m 54s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 10 new or modified test files. 0 mvndep 1m 46s Maven dependency ordering for branch +1 mvninstall 15m 36s trunk passed +1 compile 16m 56s trunk passed +1 checkstyle 2m 2s trunk passed +1 mvnsite 2m 52s trunk passed +1 mvneclipse 1m 52s trunk passed +1 findbugs 4m 32s trunk passed +1 javadoc 2m 19s trunk passed 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 2m 17s the patch passed +1 compile 14m 53s the patch passed +1 javac 14m 53s the patch passed -0 checkstyle 2m 0s root: The patch generated 2 new + 365 unchanged - 5 fixed = 367 total (was 370) +1 mvnsite 3m 20s the patch passed +1 mvneclipse 2m 8s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 31s the patch passed +1 javadoc 2m 34s the patch passed +1 unit 2m 32s hadoop-yarn-common in the patch passed. +1 unit 13m 5s hadoop-yarn-server-nodemanager in the patch passed. -1 unit 39m 59s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 9m 8s hadoop-mapreduce-client-app in the patch passed. +1 unit 3m 28s hadoop-mapreduce-client-hs in the patch passed. +1 asflicense 0m 41s The patch does not generate ASF License warnings. 178m 57s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart Subsystem Report/Notes Docker Image:yetus/hadoop:612578f JIRA Issue YARN-6202 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12863168/YARN-6202.004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 7f87b577b9ac 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0cab572 Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/15616/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/15616/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15616/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/15616/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
            hadoopqa Hadoop QA added a comment -
            +1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 18s Docker mode activated.
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 10 new or modified test files.
            0 mvndep 1m 22s Maven dependency ordering for branch
            +1 mvninstall 13m 59s trunk passed
            +1 compile 16m 32s trunk passed
            +1 checkstyle 2m 1s trunk passed
            +1 mvnsite 3m 9s trunk passed
            +1 mvneclipse 1m 55s trunk passed
            +1 findbugs 4m 45s trunk passed
            +1 javadoc 2m 16s trunk passed
            0 mvndep 0m 18s Maven dependency ordering for patch
            +1 mvninstall 2m 32s the patch passed
            +1 compile 14m 50s the patch passed
            +1 javac 14m 50s the patch passed
            -0 checkstyle 2m 52s root: The patch generated 2 new + 365 unchanged - 5 fixed = 367 total (was 370)
            +1 mvnsite 5m 8s the patch passed
            +1 mvneclipse 3m 30s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 findbugs 5m 38s the patch passed
            +1 javadoc 2m 36s the patch passed
            +1 unit 2m 36s hadoop-yarn-common in the patch passed.
            +1 unit 13m 26s hadoop-yarn-server-nodemanager in the patch passed.
            +1 unit 39m 25s hadoop-yarn-server-resourcemanager in the patch passed.
            +1 unit 8m 43s hadoop-mapreduce-client-app in the patch passed.
            +1 unit 3m 27s hadoop-mapreduce-client-hs in the patch passed.
            +1 asflicense 0m 41s The patch does not generate ASF License warnings.
            177m 15s



            Subsystem Report/Notes
            Docker Image:yetus/hadoop:612578f
            JIRA Issue YARN-6202
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12863168/YARN-6202.004.patch
            Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
            uname Linux 0a1cbcbf6eec 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
            git revision trunk / 0cab572
            Default Java 1.8.0_121
            findbugs v3.0.0
            checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/15617/artifact/patchprocess/diff-checkstyle-root.txt
            Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15617/testReport/
            modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: .
            Console output https://builds.apache.org/job/PreCommit-YARN-Build/15617/console
            Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 10 new or modified test files. 0 mvndep 1m 22s Maven dependency ordering for branch +1 mvninstall 13m 59s trunk passed +1 compile 16m 32s trunk passed +1 checkstyle 2m 1s trunk passed +1 mvnsite 3m 9s trunk passed +1 mvneclipse 1m 55s trunk passed +1 findbugs 4m 45s trunk passed +1 javadoc 2m 16s trunk passed 0 mvndep 0m 18s Maven dependency ordering for patch +1 mvninstall 2m 32s the patch passed +1 compile 14m 50s the patch passed +1 javac 14m 50s the patch passed -0 checkstyle 2m 52s root: The patch generated 2 new + 365 unchanged - 5 fixed = 367 total (was 370) +1 mvnsite 5m 8s the patch passed +1 mvneclipse 3m 30s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 38s the patch passed +1 javadoc 2m 36s the patch passed +1 unit 2m 36s hadoop-yarn-common in the patch passed. +1 unit 13m 26s hadoop-yarn-server-nodemanager in the patch passed. +1 unit 39m 25s hadoop-yarn-server-resourcemanager in the patch passed. +1 unit 8m 43s hadoop-mapreduce-client-app in the patch passed. +1 unit 3m 27s hadoop-mapreduce-client-hs in the patch passed. +1 asflicense 0m 41s The patch does not generate ASF License warnings. 177m 15s Subsystem Report/Notes Docker Image:yetus/hadoop:612578f JIRA Issue YARN-6202 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12863168/YARN-6202.004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 0a1cbcbf6eec 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0cab572 Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/15617/artifact/patchprocess/diff-checkstyle-root.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/15617/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/15617/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.

            LGTM. Would you mind filing newbie JIRAs to resolve those two checkstyle issues?

            templedf Daniel Templeton added a comment - LGTM. Would you mind filing newbie JIRAs to resolve those two checkstyle issues?
            yufeigu Yufei Gu added a comment -
            yufeigu Yufei Gu added a comment - Filed MAPREDUCE-6878 and YARN-6497 .

            Thanks for the patch, yufeigu, and, haibochen, for the review. Committed to trunk. If you want it in branch-2, I'll need another patch for it.

            templedf Daniel Templeton added a comment - Thanks for the patch, yufeigu , and, haibochen , for the review. Committed to trunk. If you want it in branch-2, I'll need another patch for it.
            hudson Hudson added a comment -

            SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11610 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11610/)
            YARN-6202. Configuration item Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY is (templedf: rev dd43b895c2e50fa97cb7327be77509b87dad1823)

            • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/event/DrainDispatcher.java
            • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
            • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestFSRMStateStore.java
            • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java
            • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/DummyContainerManager.java
            • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStore.java
            • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java
            • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMDispatcher.java
            • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestAbstractYarnScheduler.java
            • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceLocalizationService.java
            • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
            • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestLeveldbRMStateStore.java
            • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
            • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManagerRecovery.java
            • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
            • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/Dispatcher.java
            • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/EventDispatcher.java
            hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11610 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11610/ ) YARN-6202 . Configuration item Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY is (templedf: rev dd43b895c2e50fa97cb7327be77509b87dad1823) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/event/DrainDispatcher.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestFSRMStateStore.java (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistoryServer.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/DummyContainerManager.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestZKRMStateStore.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMDispatcher.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestAbstractYarnScheduler.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceLocalizationService.java (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestLeveldbRMStateStore.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManagerRecovery.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/Dispatcher.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/EventDispatcher.java
            yufeigu Yufei Gu added a comment -

            Thanks templedf for the review and commit. Thanks haibochen for the review. I saw other projects use these configuration items, better to not commit this to branch2.

            yufeigu Yufei Gu added a comment - Thanks templedf for the review and commit. Thanks haibochen for the review. I saw other projects use these configuration items, better to not commit this to branch2.
            junping_du Junping Du added a comment -

            Agree. An example is: TEZ-2049. Let's keep the patch in trunk only.

            junping_du Junping Du added a comment - Agree. An example is: TEZ-2049 . Let's keep the patch in trunk only.

            Option 1 sounds good to me. Any thoughts? Vinod Kumar Vavilapalli

            I saw other projects use these configuration items, better to not commit this to branch2.

            Agree. An example is: TEZ-2049. Let's keep the patch in trunk only.

            Folks, a bunch of downstream are using this. This is not the kind of cleanup that will balance out code-cleanup vs downstream pain. Let's just revert the removals in the patch and instead deprecate the config as well as the constants.

            vinodkv Vinod Kumar Vavilapalli added a comment - Option 1 sounds good to me. Any thoughts? Vinod Kumar Vavilapalli I saw other projects use these configuration items, better to not commit this to branch2. Agree. An example is: TEZ-2049 . Let's keep the patch in trunk only. Folks, a bunch of downstream are using this. This is not the kind of cleanup that will balance out code-cleanup vs downstream pain. Let's just revert the removals in the patch and instead deprecate the config as well as the constants.
            junping_du Junping Du added a comment -

            Let's just revert the removals in the patch and instead deprecate the config as well as the constants.

            +1. That sounds like a good plan.

            junping_du Junping Du added a comment - Let's just revert the removals in the patch and instead deprecate the config as well as the constants. +1. That sounds like a good plan.

            I hunted around, and all I could find was Tez and Tajo. Tajo hasn't been touched in 3 years, and after TEZ-2049 it's no longer used by TEZ. In fact, this patch is almost the same as the what was done in TEZ-2049. Do you know of other projects that are using it, vinodkv?

            templedf Daniel Templeton added a comment - I hunted around, and all I could find was Tez and Tajo. Tajo hasn't been touched in 3 years, and after TEZ-2049 it's no longer used by TEZ. In fact, this patch is almost the same as the what was done in TEZ-2049 . Do you know of other projects that are using it, vinodkv ?
            zhengchenyu Chenyu Zheng added a comment -

            yufeigu I agree that exitOnDispatchException should be true in most scene. But some unimportant component, for examaple dispatcher in CommonNodeLabelsManager. I think that set exitOnDispatchException to true is unnecessary. NodeLabel command failed, should not make resourcemanager crash.Maybe I think we should provide a configuration item to user.

            zhengchenyu Chenyu Zheng added a comment - yufeigu I agree that exitOnDispatchException should be true in most scene. But some unimportant component, for examaple dispatcher in CommonNodeLabelsManager. I think that set exitOnDispatchException to true is unnecessary. NodeLabel command failed, should not make resourcemanager crash.Maybe I think we should provide a configuration item to user.

            People

              yufeigu Yufei Gu
              yufeigu Yufei Gu
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: