Details
-
Sub-task
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
None
-
None
-
Reviewed
Description
When a RPC quota is removed (using LIMIT => 'NONE'), the entry from hbase:quota table is not completely removed. For e.g. see below:
hbase(main):005:0> create 't2','cf1' Created table t2 Took 0.8000 seconds => Hbase::Table - t2 hbase(main):006:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT => '10M/sec' Took 0.1024 seconds hbase(main):007:0> list_quotas OWNER QUOTAS TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE 1 row(s) Took 0.0622 seconds hbase(main):008:0> scan 'hbase:quota' ROW COLUMN+CELL t.t2 column=q:s, timestamp=1531513014463, value=PBUF\x12\x0B\x12\x09\x08\x04\x10\x80\x80\x80 \x05 \x02 1 row(s) Took 0.0453 seconds hbase(main):009:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT => 'NONE' Took 0.0097 seconds hbase(main):010:0> list_quotas OWNER QUOTAS 0 row(s) Took 0.0338 seconds hbase(main):011:0> scan 'hbase:quota' ROW COLUMN+CELL t.t2 column=q:s, timestamp=1531513039505, value=PBUF\x12\x00 1 row(s) Took 0.0066 seconds
Attachments
Attachments
- hbase-20885.master.001.patch
- 11 kB
- Sakthi
- hbase-20885.master.002.patch
- 10 kB
- Sakthi
- hbase-20885.master.003.patch
- 13 kB
- Sakthi
- hbase-20885.master.003.patch
- 13 kB
- Sakthi
- hbase-20885.master.004.patch
- 13 kB
- Sakthi
- hbase-20885.master.005.patch
- 13 kB
- Sakthi
Issue Links
- links to
Activity
In GlobalQuotaSettingsImpl#merge :
validateQuotaTarget(other); // Propagate the Throttle QuotaProtos.Throttle.Builder throttleBuilder = (throttleProto == null ? null : throttleProto.toBuilder()); if (other instanceof ThrottleSettings) { if (throttleBuilder == null) { throttleBuilder = QuotaProtos.Throttle.newBuilder(); } ThrottleSettings otherThrottle = (ThrottleSettings) other; if (otherThrottle.proto.hasType()) { QuotaProtos.ThrottleRequest otherProto = otherThrottle.proto; if (otherProto.hasTimedQuota()) { if (otherProto.hasTimedQuota()) { validateTimedQuota(otherProto.getTimedQuota()); } switch (otherProto.getType()) { ... } else { throttleBuilder = clearThrottleBuilder(throttleBuilder); } } else { throttleBuilder = clearThrottleBuilder(throttleBuilder); } }
following solves the issue:
- throttleBuilder = clearThrottleBuilder(throttleBuilder);
+ throttleBuilder = null;
Testing done:
- The way of reproducing the issue, which is mentioned in the description, now no longer produces the issue.
- Test cases added: TestQuotaAdmin#testSetGetRemoveRPCQuota, TestQuotaAdmin#testSetModifyRemoveRPCQuota
having this messes up when a Space quota is being set up on the same table
Forgive me, how does this affect space quotas? It's not clear to me just by looking at the parent (multiple issues mentioned there).
I'm not sure if this was done on purpose (for some kind of an optimization)
99% sure this was not on purpose. GlobalQuotaSettingsImpl was me trying to consolidate rpc and space quotas together. No reason to keep around a record in hbase:quota if we have no corresponding quota. Let me look more closely at your patch, seems OK on a glance.
Thanks for the prompt replies elserj. Will get back to you on explaining what I meant by this "affecting space quotas".
elserj, FYI. I am yet to figure out why this happens but in the course of that, I saw few issues (HBASE-20813 & HBASE-20885) and decided to work upon these first. But, Once these 2 issues are solved, HBASE-20705(which is the parent issue) resolves. i.e. making sure that removal of table/namespace removes the RPC quotas as well & removal of the empty row after removal of an rpc quota automatically seemed to resolve HBASE-20705. I tested that myself.
hbase(main):001:0> create 't2','cf1' Created table t2 Took 1.0745 seconds => Hbase::Table - t2 hbase(main):002:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT => '10M/sec' Took 0.0328 seconds hbase(main):003:0> list_quotas OWNER QUOTAS TABLE => t2 TYPE => THROTTLE, THROTTLE_TYPE => REQUEST_SIZE, LIMIT => 10M/sec, SCOPE => MACHINE 1 row(s) Took 0.1078 seconds hbase(main):004:0> scan 'hbase:quota' ROW COLUMN+CELL t.t2 column=q:s, timestamp=1532714834471, value=PBUF\x12\x0B\x12\x09\x08\x04\x10\x80\x80\x80\x05 \x02 1 row(s) Took 0.0301 seconds hbase(main):005:0> set_quota TYPE => THROTTLE, TABLE => 't2', LIMIT => 'NONE' Took 0.0066 seconds hbase(main):006:0> list_quotas OWNER QUOTAS 0 row(s) Took 0.0248 seconds hbase(main):007:0> scan 'hbase:quota' ROW COLUMN+CELL t.t2 column=q:s, timestamp=1532714855300, value=PBUF\x12\x00 1 row(s) Took 0.0034 seconds hbase(main):008:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '1G', POLICY => NO_WRITES Took 0.0145 seconds hbase(main):009:0> list_quotas OWNER QUOTAS TABLE => t2 TYPE => SPACE, TABLE => t2, LIMIT => 1073741824, VIOLATION_POLICY => NO_WRITES 1 row(s) Took 0.0340 seconds hbase(main):010:0> scan 'hbase:quota' ROW COLUMN+CELL t.t2 column=q:s, timestamp=1532714870451, value=PBUF\x12\x00\x1A\x08\x08\x80\x80\x80\x80\x04\x10\x03 1 row(s) Took 0.0045 seconds hbase(main):011:0> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => 'NONE', POLICY => NO_WRITES Took 0.0075 seconds hbase(main):012:0> list_quotas OWNER QUOTAS TABLE => t2 TYPE => SPACE, TABLE => t2, REMOVE => true 1 row(s) Took 0.0353 seconds hbase(main):013:0> scan 'hbase:quota' ROW COLUMN+CELL t.t2 column=q:s, timestamp=1532714887189, value=PBUF\x12\x00\x1A\x02\x18\x01 1 row(s)
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 0s | Docker mode activated. |
-1 | docker | 0m 3s | Docker failed to build yetus/hbase:b002b0b. |
Subsystem | Report/Notes |
---|---|
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12933665/hbase-20885.master.003.patch |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/13856/console |
Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
elserj, there seems to be an issue with the qa? Didn't trigger till first .003 patches, had to upload .003 again to trigger the above qa. I didn't know how to go about the result (docker failure), so tried uploading .004 with same content to see if qa throws up same error, but it hasn't started yet.
Seems like something is busted. https://builds.apache.org/job/PreCommit-HBASE-Build/13857/console was kicked off, but couldnt' clone the git repo.
I don't think it's anything wrong with your change, but we'll want to get a QA run to make sure other quota stuff isn't broken.
Makes sense elserj. And the way to go about it would be? ( is there a way for manual triggering of the QA? or waiting for auto-kick off is the only solution?)
I filed INFRA-16845 as it seems like a host-level issue and just re-submitted your job. Anyone who is a committer on any ASF project can get a Jenkins account and manually submit the job.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 9s | Docker mode activated. |
Prechecks | |||
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
master Compile Tests | |||
0 | mvndep | 0m 25s | Maven dependency ordering for branch |
+1 | mvninstall | 4m 55s | master passed |
+1 | compile | 2m 40s | master passed |
+1 | checkstyle | 1m 51s | master passed |
+1 | shadedjars | 4m 38s | branch has no errors when building our shaded downstream artifacts. |
+1 | findbugs | 3m 29s | master passed |
+1 | javadoc | 0m 51s | master passed |
Patch Compile Tests | |||
0 | mvndep | 0m 14s | Maven dependency ordering for patch |
+1 | mvninstall | 4m 55s | the patch passed |
+1 | compile | 2m 30s | the patch passed |
+1 | javac | 2m 30s | the patch passed |
-1 | checkstyle | 0m 33s | hbase-client: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) |
-1 | checkstyle | 1m 11s | hbase-server: The patch generated 42 new + 3 unchanged - 0 fixed = 45 total (was 3) |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedjars | 4m 34s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 10m 8s | Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0. |
+1 | findbugs | 3m 15s | the patch passed |
+1 | javadoc | 0m 49s | the patch passed |
Other Tests | |||
+1 | unit | 3m 5s | hbase-client in the patch passed. |
-1 | unit | 20m 13s | hbase-server in the patch failed. |
+1 | asflicense | 0m 23s | The patch does not generate ASF License warnings. |
71m 28s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hbase.quotas.TestGlobalQuotaSettingsImpl |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12933672/hbase-20885.master.004.patch |
Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
uname | Linux 8f3300c22f2b 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27 10:45:36 UTC 2018 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh |
git revision | master / a8e184dc77 |
maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
Default Java | 1.8.0_171 |
findbugs | v3.1.0-RC3 |
checkstyle | https://builds.apache.org/job/PreCommit-HBASE-Build/13865/artifact/patchprocess/diff-checkstyle-hbase-client.txt |
checkstyle | https://builds.apache.org/job/PreCommit-HBASE-Build/13865/artifact/patchprocess/diff-checkstyle-hbase-server.txt |
unit | https://builds.apache.org/job/PreCommit-HBASE-Build/13865/artifact/patchprocess/patch-unit-hbase-server.txt |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/13865/testReport/ |
Max. process+thread count | 650 (vs. ulimit of 10000) |
modules | C: hbase-client hbase-server U: . |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/13865/console |
Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
jatsakthi, looks like you have some test failures to figure out.
[INFO] Running org.apache.hadoop.hbase.quotas.TestGlobalQuotaSettingsImpl [ERROR] Tests run: 3, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.204 s <<< FAILURE! - in org.apache.hadoop.hbase.quotas.TestGlobalQuotaSettingsImpl [ERROR] testMergeThrottleAndSpace(org.apache.hadoop.hbase.quotas.TestGlobalQuotaSettingsImpl) Time elapsed: 0.019 s <<< ERROR! java.lang.NullPointerException at org.apache.hadoop.hbase.quotas.TestGlobalQuotaSettingsImpl.testMergeThrottleAndSpace(TestGlobalQuotaSettingsImpl.java:114) [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.22 s <<< FAILURE! - in org.apache.hadoop.hbase.quotas.TestGlobalQuotaSettingsImpl [ERROR] testMergeThrottleAndSpace(org.apache.hadoop.hbase.quotas.TestGlobalQuotaSettingsImpl) Time elapsed: 0 s <<< ERROR! java.lang.NullPointerException at org.apache.hadoop.hbase.quotas.TestGlobalQuotaSettingsImpl.testMergeThrottleAndSpace(TestGlobalQuotaSettingsImpl.java:114) [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.237 s <<< FAILURE! - in org.apache.hadoop.hbase.quotas.TestGlobalQuotaSettingsImpl [ERROR] testMergeThrottleAndSpace(org.apache.hadoop.hbase.quotas.TestGlobalQuotaSettingsImpl) Time elapsed: 0.003 s <<< ERROR! java.lang.NullPointerException at org.apache.hadoop.hbase.quotas.TestGlobalQuotaSettingsImpl.testMergeThrottleAndSpace(TestGlobalQuotaSettingsImpl.java:114)
Please also fix the checkstyle in the next patch. Thanks!
+1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 14s | Docker mode activated. |
Prechecks | |||
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
master Compile Tests | |||
0 | mvndep | 0m 26s | Maven dependency ordering for branch |
+1 | mvninstall | 5m 46s | master passed |
+1 | compile | 2m 46s | master passed |
+1 | checkstyle | 1m 55s | master passed |
+1 | shadedjars | 5m 5s | branch has no errors when building our shaded downstream artifacts. |
+1 | findbugs | 3m 43s | master passed |
+1 | javadoc | 1m 4s | master passed |
Patch Compile Tests | |||
0 | mvndep | 0m 17s | Maven dependency ordering for patch |
+1 | mvninstall | 5m 41s | the patch passed |
+1 | compile | 2m 59s | the patch passed |
+1 | javac | 2m 59s | the patch passed |
+1 | checkstyle | 1m 56s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedjars | 5m 2s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 11m 17s | Patch does not cause any errors with Hadoop 2.7.4 or 3.0.0. |
+1 | findbugs | 3m 22s | the patch passed |
+1 | javadoc | 0m 53s | the patch passed |
Other Tests | |||
+1 | unit | 3m 0s | hbase-client in the patch passed. |
+1 | unit | 124m 46s | hbase-server in the patch passed. |
+1 | asflicense | 0m 46s | The patch does not generate ASF License warnings. |
181m 49s |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12934009/hbase-20885.master.005.patch |
Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
uname | Linux bc92e2f14413 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27 10:45:36 UTC 2018 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh |
git revision | master / 78164efcf4 |
maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
Default Java | 1.8.0_171 |
findbugs | v3.1.0-RC3 |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/13893/testReport/ |
Max. process+thread count | 4389 (vs. ulimit of 10000) |
modules | C: hbase-client hbase-server U: . |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/13893/console |
Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
stack, Apache9, looking at bringing this back to your respective RM'ing branches. Please advise if "no" (low risk, change limited to quotas, good testing coverage here)
Thanks, Duo. Pushed to master, branch-2, and branch-2.1 for now.
Will wait on a reply from stack prior to branch-2.0
Results for branch branch-2.0
build #628 on builds.a.o: -1 overall
details (if available):
+1 general checks
– For more information see general report
-1 jdk8 hadoop2 checks
– For more information see jdk8 (hadoop2) report
-1 jdk8 hadoop3 checks
– For more information see jdk8 (hadoop3) report
+1 source release artifact
– See build output for details.
Results for branch branch-2
build #1062 on builds.a.o: -1 overall
details (if available):
+1 general checks
– For more information see general report
-1 jdk8 hadoop2 checks
– For more information see jdk8 (hadoop2) report
-1 jdk8 hadoop3 checks
– For more information see jdk8 (hadoop3) report
+1 source release artifact
– See build output for details.
+1 client integration test
Results for branch branch-2.1
build #140 on builds.a.o: -1 overall
details (if available):
+1 general checks
– For more information see general report
-1 jdk8 hadoop2 checks
– For more information see jdk8 (hadoop2) report
-1 jdk8 hadoop3 checks
– For more information see jdk8 (hadoop3) report
+1 source release artifact
– See build output for details.
+1 client integration test
Results for branch master
build #418 on builds.a.o: -1 overall
details (if available):
-1 general checks
– For more information see general report
-1 jdk8 hadoop2 checks
– For more information see jdk8 (hadoop2) report
-1 jdk8 hadoop3 checks
– For more information see jdk8 (hadoop3) report
+1 source release artifact
– See build output for details.
+1 client integration test
I'm not sure if this was done on purpose (for some kind of an optimization), but having this messes up when a Space quota is being set up on the same table (See parent issue for reference).