Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.0.0-beta-1
    • 2.0.0-beta-2, 2.0.0
    • test
    • None
    • Reviewed

    Description

      TestCoprocessorWhitelistMasterObserver is failing 33% of the time. In the logs it looks like the failure is related to Master initialization.

      Following log is from https://builds.apache.org/job/HBase%20Nightly/job/branch-2/203 

      2018-01-26 02:36:36,686 WARN [M:0;1f0c4777c1ba:35049] master.TableNamespaceManager(307): Caught exception in initializing namespace table manager
      org.apache.hadoop.hbase.DoNotRetryIOException: hconnection-0x18cd2ac8 closed
      at org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegion(ConnectionImplementation.java:722)
      at org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.locateRegion(ConnectionUtils.java:131)
      at org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegion(ConnectionImplementation.java:714)
      at org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.locateRegion(ConnectionUtils.java:131)
      at org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegion(ConnectionImplementation.java:684)
      at org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.locateRegion(ConnectionUtils.java:131)
      at org.apache.hadoop.hbase.client.ConnectionImplementation.getRegionLocation(ConnectionImplementation.java:562)
      at org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.getRegionLocation(ConnectionUtils.java:131)
      at org.apache.hadoop.hbase.client.HRegionLocator.getRegionLocation(HRegionLocator.java:73)
      at org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:223)
      at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:105)
      at org.apache.hadoop.hbase.client.HTable.get(HTable.java:388)
      at org.apache.hadoop.hbase.client.HTable.get(HTable.java:362)
      at org.apache.hadoop.hbase.master.TableNamespaceManager.get(TableNamespaceManager.java:141)
      at org.apache.hadoop.hbase.master.TableNamespaceManager.isTableAvailableAndInitialized(TableNamespaceManager.java:281)
      at org.apache.hadoop.hbase.master.TableNamespaceManager.start(TableNamespaceManager.java:103)
      at org.apache.hadoop.hbase.master.ClusterSchemaServiceImpl.doStart(ClusterSchemaServiceImpl.java:62)
      at org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.startAsync(AbstractService.java:226)
      at org.apache.hadoop.hbase.master.HMaster.initClusterSchemaService(HMaster.java:1059)
      at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:921)
      at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2034)
      at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:553)
      at java.lang.Thread.run(Thread.java:748)
      2018-01-26 02:36:36,691 ERROR [M:0;1f0c4777c1ba:35049] helpers.MarkerIgnoringBase(159): Failed to become active master
      java.lang.IllegalStateException: Expected the service ClusterSchemaServiceImpl [FAILED] to be RUNNING, but the service has FAILED
      at org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.checkCurrentState(AbstractService.java:345)
      at org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.awaitRunning(AbstractService.java:291)
      at org.apache.hadoop.hbase.master.HMaster.initClusterSchemaService(HMaster.java:1061)
      at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:921)
      at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2034)
      at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:553)
      at java.lang.Thread.run(Thread.java:748)
      Caused by: org.apache.hadoop.hbase.DoNotRetryIOException: hconnection-0x18cd2ac8 closed
      at org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegion(ConnectionImplementation.java:722)
      at org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.locateRegion(ConnectionUtils.java:131)
      at org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegion(ConnectionImplementation.java:714)
      at org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.locateRegion(ConnectionUtils.java:131)
      at org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegion(ConnectionImplementation.java:684)
      at org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.locateRegion(ConnectionUtils.java:131)
      at org.apache.hadoop.hbase.client.ConnectionImplementation.getRegionLocation(ConnectionImplementation.java:562)
      at org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.getRegionLocation(ConnectionUtils.java:131)
      at org.apache.hadoop.hbase.client.HRegionLocator.getRegionLocation(HRegionLocator.java:73)
      at org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:223)
      at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:105)
      at org.apache.hadoop.hbase.client.HTable.get(HTable.java:388)
      at org.apache.hadoop.hbase.client.HTable.get(HTable.java:362)
      at org.apache.hadoop.hbase.master.TableNamespaceManager.get(TableNamespaceManager.java:141)
      at org.apache.hadoop.hbase.master.TableNamespaceManager.isTableAvailableAndInitialized(TableNamespaceManager.java:281)
      at org.apache.hadoop.hbase.master.TableNamespaceManager.start(TableNamespaceManager.java:103)
      at org.apache.hadoop.hbase.master.ClusterSchemaServiceImpl.doStart(ClusterSchemaServiceImpl.java:62)
      at org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.startAsync(AbstractService.java:226)
      at org.apache.hadoop.hbase.master.HMaster.initClusterSchemaService(HMaster.java:1059)
      ... 4 more
      2018-01-26 02:36:36,691 ERROR [M:0;1f0c4777c1ba:35049] helpers.MarkerIgnoringBase(143): Master server abort: loaded coprocessors are: [org.apache.hadoop.hbase.security.access.TestCoprocessorWhitelistMasterObserver$TestRegionObserver, org.apache.hadoop.hbase.security.access.CoprocessorWhitelistMasterObserver, org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint]
      2018-01-26 02:36:36,691 ERROR [M:0;1f0c4777c1ba:35049] helpers.MarkerIgnoringBase(159): ***** ABORTING master 1f0c4777c1ba,35049,1516934184742: Unhandled exception. Starting shutdown. *****
      java.lang.IllegalStateException: Expected the service ClusterSchemaServiceImpl [FAILED] to be RUNNING, but the service has FAILED
      at org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.checkCurrentState(AbstractService.java:345)
      at org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.awaitRunning(AbstractService.java:291)
      at org.apache.hadoop.hbase.master.HMaster.initClusterSchemaService(HMaster.java:1061)
      at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:921)
      at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2034)
      at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:553)
      at java.lang.Thread.run(Thread.java:748)
      Caused by: org.apache.hadoop.hbase.DoNotRetryIOException: hconnection-0x18cd2ac8 closed
      at org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegion(ConnectionImplementation.java:722)
      at org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.locateRegion(ConnectionUtils.java:131)
      at org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegion(ConnectionImplementation.java:714)
      at org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.locateRegion(ConnectionUtils.java:131)
      at org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegion(ConnectionImplementation.java:684)
      at org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.locateRegion(ConnectionUtils.java:131)
      at org.apache.hadoop.hbase.client.ConnectionImplementation.getRegionLocation(ConnectionImplementation.java:562)
      at org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.getRegionLocation(ConnectionUtils.java:131)
      at org.apache.hadoop.hbase.client.HRegionLocator.getRegionLocation(HRegionLocator.java:73)
      at org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:223)
      at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:105)
      at org.apache.hadoop.hbase.client.HTable.get(HTable.java:388)
      at org.apache.hadoop.hbase.client.HTable.get(HTable.java:362)
      at org.apache.hadoop.hbase.master.TableNamespaceManager.get(TableNamespaceManager.java:141)
      at org.apache.hadoop.hbase.master.TableNamespaceManager.isTableAvailableAndInitialized(TableNamespaceManager.java:281)
      at org.apache.hadoop.hbase.master.TableNamespaceManager.start(TableNamespaceManager.java:103)
      at org.apache.hadoop.hbase.master.ClusterSchemaServiceImpl.doStart(ClusterSchemaServiceImpl.java:62)
      at org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.startAsync(AbstractService.java:226)
      at org.apache.hadoop.hbase.master.HMaster.initClusterSchemaService(HMaster.java:1059)
      ... 4 more
      2018-01-26 02:36:36,692 DEBUG [M:0;1f0c4777c1ba:35049] coprocessor.CoprocessorHost(289): Stop coprocessor org.apache.hadoop.hbase.security.access.CoprocessorWhitelistMasterObserver
      2018-01-26 02:36:36,692 WARN [M:0;1f0c4777c1ba:35049] coprocessor.BaseEnvironment(99): Not stopping coprocessor org.apache.hadoop.hbase.security.access.CoprocessorWhitelistMasterObserver because not active (state=STOPPED)
      2018-01-26 02:36:36,692 INFO [M:0;1f0c4777c1ba:35049] regionserver.HRegionServer(2142): ***** STOPPING region server '1f0c4777c1ba,35049,1516934184742' *****

      Attachments

        1. HBASE-19868.branch-2.001.patch
          3 kB
          Michael Stack
        2. HBASE-19868.master.002.patch
          2 kB
          Peter Somogyi

        Issue Links

          Activity

            stack Michael Stack added a comment -

            psomogyi How'd you extract the exception? Going about nightlies, I was unable to get good log. The exception seems to come after the log run at 

            02:36:36,686 (Log here ends at https://builds.apache.org/job/HBase%20Nightly/job/branch-2/203/testReport/junit/org.apache.hadoop.hbase.security.access/TestCoprocessorWhitelistMasterObserver/org_apache_hadoop_hbase_security_access_TestCoprocessorWhitelistMasterObserver/  2018-01-26 02:36:36,440)

            I can't make the test fail locally which I'm guessing is what you are finding.

            (I was looking to see why we are not archiving the surefire test run output seemingly. It looks like the flags are in place but i can't find the raw emissions. Need to dig more).

            So, the odd thing about this test is conf.setInt("hbase.client.retries.number", 1) (I think). If stuff is slow around startup, we'll fail our one attempt. Its happening in

            ClusterSchemaServiceImpl which is cast as a Guava Service. It is being started async. Are we stuck in 

            isTableAvailableAndInitialized failing our one attempt over and over. I can't tell. Or is it that Master fails to come up and then we are just stuck in mini cluster startup trying to scan a meta that is not coming? Let me try some debug and up the retries to 5.... Test should still fail fast....

             

            stack Michael Stack added a comment - psomogyi How'd you extract the exception? Going about nightlies, I was unable to get good log. The exception seems to come after the log run at  02:36:36,686 (Log here ends at https://builds.apache.org/job/HBase%20Nightly/job/branch-2/203/testReport/junit/org.apache.hadoop.hbase.security.access/TestCoprocessorWhitelistMasterObserver/org_apache_hadoop_hbase_security_access_TestCoprocessorWhitelistMasterObserver/   2018-01-26 02:36:36,440) I can't make the test fail locally which I'm guessing is what you are finding. (I was looking to see why we are not archiving the surefire test run output seemingly. It looks like the flags are in place but i can't find the raw emissions. Need to dig more). So, the odd thing about this test is conf.setInt("hbase.client.retries.number", 1) (I think). If stuff is slow around startup, we'll fail our one attempt. Its happening in ClusterSchemaServiceImpl which is cast as a Guava Service. It is being started async. Are we stuck in  isTableAvailableAndInitialized failing our one attempt over and over. I can't tell. Or is it that Master fails to come up and then we are just stuck in mini cluster startup trying to scan a meta that is not coming? Let me try some debug and up the retries to 5.... Test should still fail fast....  
            stack Michael Stack added a comment -

            2.001 just ups retries from 1 to 5. Test runs in about same time locally at least. Pushed to master and branch-2 to see.

            stack Michael Stack added a comment - 2.001 just ups retries from 1 to 5. Test runs in about same time locally at least. Pushed to master and branch-2 to see.
            stack Michael Stack added a comment -

            Oh, flakey 2.0 just came in. It has a few of these failures in the top-list. Thats good because these have more logs.... https://builds.apache.org/job/HBASE-Flaky-Tests-branch2.0/983/ I'm still a bit stuck though. Our thread-dumper seems to be missing in this test run.... 

             

             

            stack Michael Stack added a comment - Oh, flakey 2.0 just came in. It has a few of these failures in the top-list. Thats good because these have more logs.... https://builds.apache.org/job/HBASE-Flaky-Tests-branch2.0/983/  I'm still a bit stuck though. Our thread-dumper seems to be missing in this test run....     
            psomogyi Peter Somogyi added a comment -

            stack, I was not able to get the logs through Test Results, but I found these log entries when I downloaded the test_logs.zip from the Artifacts.

            There you can find something like this which indicates the test case was started:

            018-01-26 02:36:20,256 INFO  [Time-limited test] hbase.ResourceChecker(148): before: security.access.TestCoprocessorWhitelistMasterObserver#testCreationNonWhitelistedCoprocessorPath Thread=258, OpenFileDescriptor=2184, MaxFileDescriptor=1048576, SystemLoadAverage=1594, ProcessCount=15, AvailableMemoryMB=11745

            I can't find anything in the logs that it ever finished. For other tests you can see something like this:

            2018-01-26 02:36:20,239 INFO  [Time-limited test] hbase.ResourceChecker(172): after: security.access.TestCoprocessorWhitelistMasterObserver#testDFSNameWhitelistedWorks Thread=258 (was 232) - Thread LEAK? -, OpenFileDescriptor=2184 (was 1889) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=1048576 (was 1048576), SystemLoadAverage=1594 (was 1584) - SystemLoadAverage LEAK? -, ProcessCount=16 (was 15) - ProcessCount LEAK? -, AvailableMemoryMB=11747 (was 12147)

            I was also unable to reproduce it locally. I will try to work on this today.

            psomogyi Peter Somogyi added a comment - stack , I was not able to get the logs through Test Results, but I found these log entries when I downloaded the  test_logs.zip  from the Artifacts. There you can find something like this which indicates the test case was started: 018-01-26 02:36:20,256 INFO  [Time-limited test] hbase.ResourceChecker(148): before: security.access.TestCoprocessorWhitelistMasterObserver#testCreationNonWhitelistedCoprocessorPath Thread=258, OpenFileDescriptor=2184, MaxFileDescriptor=1048576, SystemLoadAverage=1594, ProcessCount=15, AvailableMemoryMB=11745 I can't find anything in the logs that it ever finished. For other tests you can see something like this: 2018-01-26 02:36:20,239 INFO  [Time-limited test] hbase.ResourceChecker(172): after: security.access.TestCoprocessorWhitelistMasterObserver#testDFSNameWhitelistedWorks Thread=258 (was 232) - Thread LEAK? -, OpenFileDescriptor=2184 (was 1889) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=1048576 (was 1048576), SystemLoadAverage=1594 (was 1584) - SystemLoadAverage LEAK? -, ProcessCount=16 (was 15) - ProcessCount LEAK? -, AvailableMemoryMB=11747 (was 12147) I was also unable to reproduce it locally. I will try to work on this today.
            hudson Hudson added a comment -

            FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4495 (See https://builds.apache.org/job/HBase-Trunk_matrix/4495/)
            HBASE-19868 TestCoprocessorWhitelistMasterObserver is flakey (stack: rev 98c9d8f7a1c8155d41fafed0b9ca792cc5fc7098)

            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestCoprocessorWhitelistMasterObserver.java
            hudson Hudson added a comment - FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4495 (See https://builds.apache.org/job/HBase-Trunk_matrix/4495/ ) HBASE-19868 TestCoprocessorWhitelistMasterObserver is flakey (stack: rev 98c9d8f7a1c8155d41fafed0b9ca792cc5fc7098) (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestCoprocessorWhitelistMasterObserver.java
            psomogyi Peter Somogyi added a comment -

            Looking into the latest log files the fix you pushed made the TestCoprocessorWhitelistMasterObserver#testCreationNonWhitelistedCoprocessorPath to pass. Even so TestCoprocessorWhitelistMasterObserver still failed with another test: testSubstringNonWhitelisted.

            2018-01-29 20:38:14,211 DEBUG [Time-limited test] util.JVMClusterUtil(238): Shutting down HBase Cluster
            2018-01-29 20:38:14,211 INFO [Time-limited test] regionserver.HRegionServer(2142): ***** STOPPING region server '5fbd36eb442f,33089,1517258292994' *****
            2018-01-29 20:38:14,211 INFO [Time-limited test] regionserver.HRegionServer(2156): STOPPED: Stopped by Time-limited test
            2018-01-29 20:38:14,225 DEBUG [Time-limited test-EventThread] zookeeper.ZKWatcher(443): regionserver:33119-0x16143a43d450001, quorum=localhost:53201, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master
            2018-01-29 20:38:14,225 INFO [Time-limited test] regionserver.HRegionServer(2142): ***** STOPPING region server '5fbd36eb442f,33119,1517258293639' *****
            2018-01-29 20:38:14,225 DEBUG [Time-limited test-EventThread] zookeeper.ZKWatcher(443): master:33089-0x16143a43d450000, quorum=localhost:53201, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master
            2018-01-29 20:38:14,225 INFO [Time-limited test] regionserver.HRegionServer(2156): STOPPED: Shutdown requested
            2018-01-29 20:38:14,225 INFO [Time-limited test] util.JVMClusterUtil(322): Shutdown of 1 master(s) and 1 regionserver(s) complete
            

            I think this failure is related to the newly introduced timeout. The test is categorized as Medium test, so the timeout is 180 seconds.
            First log entry for this test is at 2018-01-29 20:35:14,223 and the "Shutting down HBase Cluster" message was at 2018-01-29 20:38:14,211.

            I can upload a patch to recategorize TestCoprocessorWhitelistMasterObserver as Large test. 

            CC: Apache9

            psomogyi Peter Somogyi added a comment - Looking into the latest log files the fix you pushed made the TestCoprocessorWhitelistMasterObserver#testCreationNonWhitelistedCoprocessorPath to pass. Even so TestCoprocessorWhitelistMasterObserver still failed with another test: testSubstringNonWhitelisted. 2018-01-29 20:38:14,211 DEBUG [Time-limited test] util.JVMClusterUtil(238): Shutting down HBase Cluster 2018-01-29 20:38:14,211 INFO [Time-limited test] regionserver.HRegionServer(2142): ***** STOPPING region server '5fbd36eb442f,33089,1517258292994' ***** 2018-01-29 20:38:14,211 INFO [Time-limited test] regionserver.HRegionServer(2156): STOPPED: Stopped by Time-limited test 2018-01-29 20:38:14,225 DEBUG [Time-limited test-EventThread] zookeeper.ZKWatcher(443): regionserver:33119-0x16143a43d450001, quorum=localhost:53201, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2018-01-29 20:38:14,225 INFO [Time-limited test] regionserver.HRegionServer(2142): ***** STOPPING region server '5fbd36eb442f,33119,1517258293639' ***** 2018-01-29 20:38:14,225 DEBUG [Time-limited test-EventThread] zookeeper.ZKWatcher(443): master:33089-0x16143a43d450000, quorum=localhost:53201, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2018-01-29 20:38:14,225 INFO [Time-limited test] regionserver.HRegionServer(2156): STOPPED: Shutdown requested 2018-01-29 20:38:14,225 INFO [Time-limited test] util.JVMClusterUtil(322): Shutdown of 1 master(s) and 1 regionserver(s) complete I think this failure is related to the newly introduced timeout. The test is categorized as Medium test, so the timeout is 180 seconds. First log entry for this test is at 2018-01-29 20:35:14,223 and the "Shutting down HBase Cluster" message was at 2018-01-29 20:38:14,211. I can upload a patch to recategorize TestCoprocessorWhitelistMasterObserver as Large test.  CC: Apache9
            zhangduo Duo Zhang added a comment -

            Thanks psomogyi for the ping. I'm +1 on changing all tests which run longer than 2 minutes to LargeTests as 3 minutes is a hard limit, we need to make sure that the test will not timeout even if runs on a slow machine(like our jenkins...).

            Thanks.

            zhangduo Duo Zhang added a comment - Thanks psomogyi for the ping. I'm +1 on changing all tests which run longer than 2 minutes to LargeTests as 3 minutes is a hard limit, we need to make sure that the test will not timeout even if runs on a slow machine(like our jenkins...). Thanks.
            stack Michael Stack added a comment - - edited

            Lets try it psomogyi I pushed to master and branch-2. There is likely an issue in here around the creation of the namespace artifacts done in the async Service but hopefully it manifests again in a manner in which we can see better what is going on.

             

            Leaving this open until we can say for sure the test is fixed. Thanks.

            stack Michael Stack added a comment - - edited Lets try it psomogyi I pushed to master and branch-2. There is likely an issue in here around the creation of the namespace artifacts done in the async Service but hopefully it manifests again in a manner in which we can see better what is going on.   Leaving this open until we can say for sure the test is fixed. Thanks.
            hudson Hudson added a comment -

            FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4497 (See https://builds.apache.org/job/HBase-Trunk_matrix/4497/)
            HBASE-19868 TestCoprocessorWhitelistMasterObserver is flakey (stack: rev 194890b69492dcddb9ebdd4e398aeeffc7435fac)

            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestCoprocessorWhitelistMasterObserver.java
            hudson Hudson added a comment - FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4497 (See https://builds.apache.org/job/HBase-Trunk_matrix/4497/ ) HBASE-19868 TestCoprocessorWhitelistMasterObserver is flakey (stack: rev 194890b69492dcddb9ebdd4e398aeeffc7435fac) (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestCoprocessorWhitelistMasterObserver.java
            zhangduo Duo Zhang added a comment -

            Resolve.

            zhangduo Duo Zhang added a comment - Resolve.

            People

              psomogyi Peter Somogyi
              psomogyi Peter Somogyi
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: