Details

    Description

      The following can be observed in master branch:

      java.lang.NullPointerException
      	at org.apache.hadoop.hbase.rest.TestTableResource.setUpBeforeClass(TestTableResource.java:134)
      

      The NPE comes from the following in TestEndToEndSplitTransaction :

              compactAndBlockUntilDone(TEST_UTIL.getAdmin(),
                TEST_UTIL.getMiniHBaseCluster().getRegionServer(0), daughterA.getRegionName());
      

      Initial check of the code shows that TestEndToEndSplitTransaction uses TEST_UTIL instance which is created within TestEndToEndSplitTransaction. However, TestTableResource creates its own instance of HBaseTestingUtility.
      Meaning TEST_UTIL.getMiniHBaseCluster() would return null, since the instance created by TestEndToEndSplitTransaction has hbaseCluster as null.

      Attachments

        1. HBASE-21076.0.patch
          5 kB
          Sean Busbey
        2. HBASE-21076.1.patch
          7 kB
          Sean Busbey

        Issue Links

          Activity

            hudson Hudson added a comment -

            Results for branch branch-1.4
            build #433 on builds.a.o: -1 overall


            details (if available):

            -1 general checks
            – For more information see general report

            -1 jdk7 checks
            – For more information see jdk7 report

            -1 jdk8 hadoop2 checks
            – For more information see jdk8 (hadoop2) report

            +1 source release artifact
            – See build output for details.

            hudson Hudson added a comment - Results for branch branch-1.4 build #433 on builds.a.o : -1 overall details (if available): -1 general checks – For more information see general report -1 jdk7 checks – For more information see jdk7 report -1 jdk8 hadoop2 checks – For more information see jdk8 (hadoop2) report +1 source release artifact – See build output for details.
            hudson Hudson added a comment -

            Results for branch branch-2.0
            build #717 on builds.a.o: -1 overall


            details (if available):

            +1 general checks
            – For more information see general report

            +1 jdk8 hadoop2 checks
            – For more information see jdk8 (hadoop2) report

            -1 jdk8 hadoop3 checks
            – For more information see jdk8 (hadoop3) report

            +1 source release artifact
            – See build output for details.

            hudson Hudson added a comment - Results for branch branch-2.0 build #717 on builds.a.o : -1 overall details (if available): +1 general checks – For more information see general report +1 jdk8 hadoop2 checks – For more information see jdk8 (hadoop2) report -1 jdk8 hadoop3 checks – For more information see jdk8 (hadoop3) report +1 source release artifact – See build output for details.
            hudson Hudson added a comment -

            Results for branch branch-1
            build #428 on builds.a.o: -1 overall


            details (if available):

            -1 general checks
            – For more information see general report

            -1 jdk7 checks
            – For more information see jdk7 report

            -1 jdk8 hadoop2 checks
            – For more information see jdk8 (hadoop2) report

            -1 source release artifact
            – See build output for details.

            hudson Hudson added a comment - Results for branch branch-1 build #428 on builds.a.o : -1 overall details (if available): -1 general checks – For more information see general report -1 jdk7 checks – For more information see jdk7 report -1 jdk8 hadoop2 checks – For more information see jdk8 (hadoop2) report -1 source release artifact – See build output for details.
            hudson Hudson added a comment -

            Results for branch branch-1.3
            build #439 on builds.a.o: -1 overall


            details (if available):

            +1 general checks
            – For more information see general report

            +1 jdk7 checks
            – For more information see jdk7 report

            -1 jdk8 hadoop2 checks
            – For more information see jdk8 (hadoop2) report

            +1 source release artifact
            – See build output for details.

            hudson Hudson added a comment - Results for branch branch-1.3 build #439 on builds.a.o : -1 overall details (if available): +1 general checks – For more information see general report +1 jdk7 checks – For more information see jdk7 report -1 jdk8 hadoop2 checks – For more information see jdk8 (hadoop2) report +1 source release artifact – See build output for details.
            hudson Hudson added a comment -

            Results for branch branch-1.2
            build #443 on builds.a.o: -1 overall


            details (if available):

            +1 general checks
            – For more information see general report

            -1 jdk7 checks
            – For more information see jdk7 report

            -1 jdk8 hadoop2 checks
            – For more information see jdk8 (hadoop2) report

            +1 source release artifact
            – See build output for details.

            hudson Hudson added a comment - Results for branch branch-1.2 build #443 on builds.a.o : -1 overall details (if available): +1 general checks – For more information see general report -1 jdk7 checks – For more information see jdk7 report -1 jdk8 hadoop2 checks – For more information see jdk8 (hadoop2) report +1 source release artifact – See build output for details.
            hudson Hudson added a comment -

            Results for branch branch-2.1
            build #227 on builds.a.o: -1 overall


            details (if available):

            +1 general checks
            – For more information see general report

            -1 jdk8 hadoop2 checks
            – For more information see jdk8 (hadoop2) report

            -1 jdk8 hadoop3 checks
            – For more information see jdk8 (hadoop3) report

            +1 source release artifact
            – See build output for details.

            +1 client integration test

            hudson Hudson added a comment - Results for branch branch-2.1 build #227 on builds.a.o : -1 overall details (if available): +1 general checks – For more information see general report -1 jdk8 hadoop2 checks – For more information see jdk8 (hadoop2) report -1 jdk8 hadoop3 checks – For more information see jdk8 (hadoop3) report +1 source release artifact – See build output for details. +1 client integration test
            hudson Hudson added a comment -

            SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1152 (See https://builds.apache.org/job/HBase-1.2-IT/1152/)
            HBASE-21076 refactor TestTableResource to ask for a multi-region table (busbey: rev 524f7fbe572897f2a318eaad4865d83a49932f1a)

            • (edit) hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/TestTableResource.java
            hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1152 (See https://builds.apache.org/job/HBase-1.2-IT/1152/ ) HBASE-21076 refactor TestTableResource to ask for a multi-region table (busbey: rev 524f7fbe572897f2a318eaad4865d83a49932f1a) (edit) hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/TestTableResource.java
            hudson Hudson added a comment -

            SUCCESS: Integrated in Jenkins build HBase-1.3-IT #461 (See https://builds.apache.org/job/HBase-1.3-IT/461/)
            HBASE-21076 refactor TestTableResource to ask for a multi-region table (busbey: rev 05460c9370c175729c10b1f0ad6d91b15e5660ac)

            • (edit) hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/TestTableResource.java
            hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build HBase-1.3-IT #461 (See https://builds.apache.org/job/HBase-1.3-IT/461/ ) HBASE-21076 refactor TestTableResource to ask for a multi-region table (busbey: rev 05460c9370c175729c10b1f0ad6d91b15e5660ac) (edit) hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/TestTableResource.java
            busbey Sean Busbey added a comment -

            pushed to all branches, since even on those that didn't already have the cross-test dependency this removed a busy wait from the test.

            busbey Sean Busbey added a comment - pushed to all branches, since even on those that didn't already have the cross-test dependency this removed a busy wait from the test.
            zhangduo Duo Zhang added a comment -

            +1. The flaky dashboard is good. All green after we commit the patch to master.

            zhangduo Duo Zhang added a comment - +1. The flaky dashboard is good. All green after we commit the patch to master.
            busbey Sean Busbey added a comment -

            I was waiting for Duo's view since it's been on master. has passed the last 25 runs. will push when i get back to a computer.

            busbey Sean Busbey added a comment - I was waiting for Duo's view since it's been on master. has passed the last 25 runs. will push when i get back to a computer.

            Looks ready to go?
            Let's commit. Just ran into this on the branch-2s, still an issue there, though not on master, at least for me.

            apurtell Andrew Kyle Purtell added a comment - Looks ready to go? Let's commit. Just ran into this on the branch-2s, still an issue there, though not on master, at least for me.
            hudson Hudson added a comment -

            Results for branch master
            build #448 on builds.a.o: -1 overall


            details (if available):

            +1 general checks
            – For more information see general report

            -1 jdk8 hadoop2 checks
            – For more information see jdk8 (hadoop2) report

            -1 jdk8 hadoop3 checks
            – For more information see jdk8 (hadoop3) report

            +1 source release artifact
            – See build output for details.

            +1 client integration test

            hudson Hudson added a comment - Results for branch master build #448 on builds.a.o : -1 overall details (if available): +1 general checks – For more information see general report -1 jdk8 hadoop2 checks – For more information see jdk8 (hadoop2) report -1 jdk8 hadoop3 checks – For more information see jdk8 (hadoop3) report +1 source release artifact – See build output for details. +1 client integration test
            yuzhihong@gmail.com Ted Yu added a comment -

            Patch for master looks good me.

            Thanks for working on this, Sean.

            yuzhihong@gmail.com Ted Yu added a comment - Patch for master looks good me. Thanks for working on this, Sean.
            busbey Sean Busbey added a comment -

            Apache9 last 4 runs came back green:

            https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/master/8/artifact/dashboard.html#job_2

            let me know when there are enough green runs for you to be comfortable with me backporting.

            busbey Sean Busbey added a comment - Apache9 last 4 runs came back green: https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/master/8/artifact/dashboard.html#job_2 let me know when there are enough green runs for you to be comfortable with me backporting.
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 0s Docker mode activated.
            -1 patch 0m 6s HBASE-21076 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for help.



            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 6s HBASE-21076 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for help. Subsystem Report/Notes JIRA Issue HBASE-21076 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12936451/HBASE-21076.1.patch Console output https://builds.apache.org/job/PreCommit-HBASE-Build/14116/console Powered by Apache Yetus 0.7.0 http://yetus.apache.org This message was automatically generated.
            busbey Sean Busbey added a comment - - edited

            v1 what I pushed to master

            • fixes checkstyle problems precommit found
            • fixes comment that said "three" instead of "four" regions were requested.

            done as v0 + addendum because in my "update, test, push" cycle I forgot to commit the staged changes before pushing

            busbey Sean Busbey added a comment - - edited v1 what I pushed to master fixes checkstyle problems precommit found fixes comment that said "three" instead of "four" regions were requested. done as v0 + addendum because in my "update, test, push" cycle I forgot to commit the staged changes before pushing
            busbey Sean Busbey added a comment -

            sweet. will push to master shortly and then can backport once the flaky job has a go at running it a few times.

            busbey Sean Busbey added a comment - sweet. will push to master shortly and then can backport once the flaky job has a go at running it a few times.
            zhangduo Duo Zhang added a comment -

            Tried locally, it works for me. +1 after you fixing the checkstyle issues. Let's push it to master to see if the flaky dashboard also says OK.

            Thanks.

            zhangduo Duo Zhang added a comment - Tried locally, it works for me. +1 after you fixing the checkstyle issues. Let's push it to master to see if the flaky dashboard also says OK. Thanks.
            busbey Sean Busbey added a comment -

            both of those checkstyle warnings are easy fixes. Can do on commit or will post a new patch if anyone wants some other change.

            busbey Sean Busbey added a comment - both of those checkstyle warnings are easy fixes. Can do on commit or will post a new patch if anyone wants some other change.
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 28s Docker mode activated.
                  Prechecks
            +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
                  master Compile Tests
            +1 mvninstall 6m 39s master passed
            +1 compile 1m 15s master passed
            +1 checkstyle 0m 44s master passed
            +1 shadedjars 6m 50s branch has no errors when building our shaded downstream artifacts.
            +1 findbugs 1m 34s master passed
            +1 javadoc 0m 53s master passed
                  Patch Compile Tests
            +1 mvninstall 6m 53s the patch passed
            +1 compile 1m 19s the patch passed
            +1 javac 1m 19s the patch passed
            -1 checkstyle 0m 46s hbase-rest: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 shadedjars 6m 57s patch has no errors when building our shaded downstream artifacts.
            +1 hadoopcheck 12m 59s Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0.
            +1 findbugs 0m 49s the patch passed
            +1 javadoc 0m 18s the patch passed
                  Other Tests
            +1 unit 2m 49s hbase-rest in the patch passed.
            +1 asflicense 0m 9s The patch does not generate ASF License warnings.
            52m 19s



            Subsystem Report/Notes
            Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b
            JIRA Issue HBASE-21076
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12936431/HBASE-21076.0.patch
            Optional Tests asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
            uname Linux c145ce8bdffe 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27 10:45:36 UTC 2018 x86_64 GNU/Linux
            Build tool maven
            Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
            git revision master / bb3494134e
            maven version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z)
            Default Java 1.8.0_181
            findbugs v3.1.0-RC3
            checkstyle https://builds.apache.org/job/PreCommit-HBASE-Build/14110/artifact/patchprocess/diff-checkstyle-hbase-rest.txt
            Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/14110/testReport/
            Max. process+thread count 2206 (vs. ulimit of 10000)
            modules C: hbase-rest U: hbase-rest
            Console output https://builds.apache.org/job/PreCommit-HBASE-Build/14110/console
            Powered by Apache Yetus 0.7.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 28s Docker mode activated.       Prechecks +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       master Compile Tests +1 mvninstall 6m 39s master passed +1 compile 1m 15s master passed +1 checkstyle 0m 44s master passed +1 shadedjars 6m 50s branch has no errors when building our shaded downstream artifacts. +1 findbugs 1m 34s master passed +1 javadoc 0m 53s master passed       Patch Compile Tests +1 mvninstall 6m 53s the patch passed +1 compile 1m 19s the patch passed +1 javac 1m 19s the patch passed -1 checkstyle 0m 46s hbase-rest: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedjars 6m 57s patch has no errors when building our shaded downstream artifacts. +1 hadoopcheck 12m 59s Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0. +1 findbugs 0m 49s the patch passed +1 javadoc 0m 18s the patch passed       Other Tests +1 unit 2m 49s hbase-rest in the patch passed. +1 asflicense 0m 9s The patch does not generate ASF License warnings. 52m 19s Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b JIRA Issue HBASE-21076 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12936431/HBASE-21076.0.patch Optional Tests asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile uname Linux c145ce8bdffe 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27 10:45:36 UTC 2018 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / bb3494134e maven version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) Default Java 1.8.0_181 findbugs v3.1.0-RC3 checkstyle https://builds.apache.org/job/PreCommit-HBASE-Build/14110/artifact/patchprocess/diff-checkstyle-hbase-rest.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/14110/testReport/ Max. process+thread count 2206 (vs. ulimit of 10000) modules C: hbase-rest U: hbase-rest Console output https://builds.apache.org/job/PreCommit-HBASE-Build/14110/console Powered by Apache Yetus 0.7.0 http://yetus.apache.org This message was automatically generated.
            busbey Sean Busbey added a comment -

            note that due to HBASE-19265, precommit isn't going to run TestTableResource (it's in the untrusted list because it fails 100% of the time before this patch). reviewers should manually run the test locally to confirm it works. Doing this while doing a minimum of other things: mvn -Dtest=TestTableResource -pl hbase-rest -am package

            busbey Sean Busbey added a comment - note that due to HBASE-19265 , precommit isn't going to run TestTableResource (it's in the untrusted list because it fails 100% of the time before this patch). reviewers should manually run the test locally to confirm it works. Doing this while doing a minimum of other things: mvn -Dtest=TestTableResource -pl hbase-rest -am package
            busbey Sean Busbey added a comment -

            -v0

            • avoid calling split at all by requesting the number of regions we want at creation
            • fix a bug in how the test turns non-printable characters into a string
            busbey Sean Busbey added a comment - -v0 avoid calling split at all by requesting the number of regions we want at creation fix a bug in how the test turns non-printable characters into a string
            busbey Sean Busbey added a comment -

            showed up in nightly for all branches that got HBASE-20940 right after it landed, which added these calls to TEST_UTIL.

            we shouldn't have TestTableResource reaching into a test class in a different module like this.

            busbey Sean Busbey added a comment - showed up in nightly for all branches that got HBASE-20940 right after it landed, which added these calls to TEST_UTIL. we shouldn't have TestTableResource reaching into a test class in a different module like this.

            People

              busbey Sean Busbey
              yuzhihong@gmail.com Ted Yu
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: