Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
0.94.6, 0.95.0
-
None
-
None
-
Reviewed
-
HideThis issue introduces a few new APIs:
* HConnectionManager:
{code}
public static HConnection createConnection(Configuration conf)
public static HConnection createConnection(Configuration conf, ExecutorService pool)
{code}
* HConnection:
{code}
public HTableInterface getTable(String tableName) throws IOException
public HTableInterface getTable(byte[] tableName) throws IOException
public HTableInterface getTable(String tableName, ExecutorService pool) throws IOException
public HTableInterface getTable(byte[] tableName, ExecutorService pool) throws IOException
{code}
By default HConnectionImplementation will create an ExecutorService when needed. The ExecutorService can optionally passed be passed in.
HTableInterfaces are retrieved from the HConnection. By default the HConnection's ExecutorService is used, but optionally that can be overridden for each HTable.ShowThis issue introduces a few new APIs: * HConnectionManager: {code} public static HConnection createConnection(Configuration conf) public static HConnection createConnection(Configuration conf, ExecutorService pool) {code} * HConnection: {code} public HTableInterface getTable(String tableName) throws IOException public HTableInterface getTable(byte[] tableName) throws IOException public HTableInterface getTable(String tableName, ExecutorService pool) throws IOException public HTableInterface getTable(byte[] tableName, ExecutorService pool) throws IOException {code} By default HConnectionImplementation will create an ExecutorService when needed. The ExecutorService can optionally passed be passed in. HTableInterfaces are retrieved from the HConnection. By default the HConnection's ExecutorService is used, but optionally that can be overridden for each HTable.
Description
Update:
I now propose deprecating HTablePool and instead introduce a getTable method on HConnection and allow HConnection to manage the ThreadPool.
Initial proposal:
Here I propose a very simple TablePool.
It could be called LightHTablePool (or something - if you have a better name).
Internally it would maintain an HConnection and an Executor service and each invocation of getTable(...) would create a new HTable and close() would just close it.
In testing I find this more light weight than HTablePool and easier to monitor in terms of resources used.
It would hardly be more than a few dozen lines of code.
Attachments
Attachments
- 6580-trunk-v6.txt
- 24 kB
- Lars Hofhansl
- 6580-0.94-v2.txt
- 20 kB
- Lars Hofhansl
- 6580-0.94.txt
- 20 kB
- Lars Hofhansl
- 6580-trunk-v5.txt
- 24 kB
- Lars Hofhansl
- 6580-trunk-v4.txt
- 21 kB
- Lars Hofhansl
- 6580-trunk-v3.txt
- 22 kB
- Lars Hofhansl
- 6580-trunk-v2.txt
- 20 kB
- Lars Hofhansl
- 6580-trunk.txt
- 10 kB
- Lars Hofhansl
- HBASE-6580_v2.patch
- 7 kB
- Adrian Muraru
- HBASE-6580_v1.patch
- 10 kB
- Adrian Muraru
Issue Links
- depends upon
-
HBASE-4805 Allow better control of resource consumption in HTable
- Closed
- is related to
-
HBASE-6956 Do not return back to HTablePool closed connections
- Closed
- relates to
-
HBASE-9117 Remove HTablePool and all HConnection pooling related APIs
- Closed
-
HBASE-7463 [REST] Use "unmanaged" HConnection instead of HTablePool
- Closed
Activity
SingleConnectionPool? SharedConnectionPool?
Or just SharedConnection (because that is actually what it is)?
I can even see replacing the current HTablePool with this.
It is simpler, should be faster, and definitely more lightweight.
I can even see replacing the current HTablePool with this.
I can see this. I believe we have a lot of users of the current HTablePool, so should be API compatible including constructors.
Hmmm... HTablePool currently optionally takes an HTableInterfaceFactory. With the model proposed here, I cannot support that, since HTable creation is precisely what I need to control.
Maybe this is the right hint, though, and what I am looking for is:
class SingleConnectionHTableFactory implement HTableInterfaceFactory {
...
}
Rather than a table pool.
This is something we need in production as well.
Digging deeper I see the underlying HConnection is already shared in the current HTablePool impl. (i.e there is a single, private Configuration instance passed when HTablePool is initialised and used for all HTable instances built by pool).
What is missing is a shared ExecutorService for all htable's so I called it SharedExecutorHTableFactory
Patch attached (it's on hbase-trunk but I can easily port to 0.92 and 0.94)
That would work. Ideally I'd like to get HTablePool out of the mix. It's heavy weight and not needed (and in fact just adds overhead) if the both the connection and thread pool are shared between HTables.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12554094/HBASE-6580_v1.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
-1 javadoc. The javadoc tool appears to have generated 99 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 24 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.TestCheckTestClasses
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3360//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3360//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3360//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3360//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3360//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3360//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3360//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3360//console
This message is automatically generated.
Lars, I agree the HTablePool is overkill here for these lightweight HTable instances but we'd still need a kind of borrow/release pooling mech for them to avoid creating a new instance every time one is needed.
Not really. After HBASE-4805 you'd create a single HConnection (via HConnectionManager.createConnection(...)) and then use this for each HTable created afterwards. HTable creation is very cheap when both HConnection and ExecutorService are passed... Cheaper indeed than then the HTablePool protocol.
Maybe HConnection should just finally get a getTable(...) method.
Maybe HConnection should just finally get a getTable(...) method.
+1. Using HConnection to get an HTable seems more intuitive and more composable than our old monolithic HTable. Lightweight HTables that are easily GC'd would also be very nice. Agree that at that point we don't really need HTablePool.
For HConnection.java, please add javadoc for tableName parameter.
+ // thread executor shared by all HTableInterface instances created + // by this connection
nit: 'created by' -> 'created from'
+ } catch (InterruptedException e) { + this.hTableExecutor.shutdownNow(); + }
Please restore interrupt status above.
+ if (hTableExecutor == null) { + // shared HTable thread executor not yet initialized + synchronized (this) { + if (hTableExecutor == null) {
Why use double checked locking ? Connection would be used to create (at least) one table, right ?
Couple of things:
- Should allow passing an ExecutorService to HConnectionManager.createConnection(...). In fact I would require that now, and have that as the only option to setup the ExecutorService.
- getTable must fail if this is a "managed", i.e. not created by createConnection (check the managed flag for that). Otherwise the HTable and the HConnection will get very confused.
I love where this is going. I wanted to do this a while ago, but felt that we're not mentally ready for this
tedyu@apache.org Thanks for having a look
Why use double checked locking ? Connection would be used to create (at least) one table, right ?
We want to have multiple HTable instances sharing the same HConnection (this) and the same ExecutorService so I ensure only one executor is ever instantiated
Should allow passing an ExecutorService to HConnectionManager.createConnection(...). In fact I would require that now, and have that as the only option to setup the ExecutorService.
Managing the shared ExecutorService internally would be more usable for the user in my opinion, no need to create or shutdown - I look at this executor as an internal detail used to execute some table operations in parallel.
getTable must fail if this is a "managed", i.e. not created by createConnection (check the managed flag for that). Otherwise the HTable and the HConnection will get very confused.
Can you elaborate? Why is not advisable to use a "managed" connection when creating a HTable.
I can do today :
HTable t = new HTable(tableName, HConnectionManager.getConnection(conf), pool)
and have an HTable using a "managed" connection
You will never be able to size the ExecutorService correctly without knowing the app. This pool is used by all HTable instances using this Connection to parallelize their requests to the RegionServers.
I'd rather have people think about it, rather than having a default that will likely not be right for their use case.
Starting with single core thread is not right. There needs to be at least an option to pass in a custom ExecutorService.
I can do today: HTable t = new HTable(tableName, HConnectionManager.getConnection(conf), pool)
You can do that, and it might even work for a bit
HConnectionManager.getConnection(...) is meant for cases where the connection is created (and potentially cached) on behalf of an HTable.
I suppose that constructor should throw an exception is the connection is managed.
This stuff is very confusing about our client.
Like HTablePool, HConnectionManager.getConnection should just go away.
Also as part of this, I would like to propose to remove HTablePool for good.
(You do not have to do this part Andrian, unless you like to , I'll happily do an addon patch.
Lars, you're right, having a way to pass an already fine tuned executor is useful such cases. building one from configuration params might not be enough. Thinking of this, I'm more in favour of adding executor param to #getTable In order to Alloa mixt workloads to share the HConnection placement info
ExecutorService lowPriority=....
ExecutorService highPriority=....
HC conn= HCM.createConnection()
conn.getTable(table1, lowPriority)
conn.getTable(table2, highPriority)
+1 removing HTablePool
+1 removing managed HConnections - is this doable? Do we use it elsewhere?
+1 keep only lightweight HTable ctors
w.r.t. removing HTablePool, is there discussion / notification on user mail list ?
renaming the issue title would be good as well; to reflect the changes planned : something like "Remove HTablePool and replace it with lightweight HTable" would grab the necessary attn I guess
Hmm... HConnection.getTable(byte[], ExecutorService) is not that much better than new HTable(byte[], HConnection, ExecutorService).
Note that a single HTable can use many threads to parallelize update requests to multiple region servers.
Let's do this?
- optionally allow createConnection to be passed a ExecutorService, I think we can document that this service will be closed when the connection is closed. If none is passed create the executor as you do (but maybe with more core threads that do not time out?).
- optionally allow getTable to override the ExecutorService.
(For example: In our case we'll have an ExecutorService with something like 100 core threads, a wait queue of size 100, and max size of 200... For a total of 300 outstanding update requests across all HTable instances).
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12555580/HBASE-6580_v2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 8 new or modified tests.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
-1 javadoc. The javadoc tool appears to have generated 107 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 23 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3520//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3520//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3520//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3520//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3520//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3520//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3520//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3520//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3520//console
This message is automatically generated.
I want to. It is hard to create a method for method replacement, though.
We could say HTablePool be gone in 0.96, or we could make a best effort replacement, that would let the majority of people recompile their existing code for 0.96 without changes.
lhofhansl Yeah, the latter suggestion would be the grown-up thing to do. Make new, critical issue?
Here's the proposed new API:
- HConnectionManager:
public static HConnection createConnection(Configuration conf) public static HConnection createConnection(Configuration conf, ExecutorService pool)
- HConnection:
public HTableInterface getTable(byte[] tableName) throws IOException { public HTableInterface getTable(byte[] tableName, ExecutorService pool) throws IOException { public HTableInterface getTable(String tableName) throws IOException {
By default HConnectionImplementation will create an ExecutorService when needed. The ExecutorService can optionally passed be passed in.
HTableInterfaces are retrieved from the HConnection. By default the HConnection's ExecutorService is used, but optionally that can be overridden for each HTable.
I'll like to get consensus here first. Then I'll post the new API to the dev list to get more feedback.
Could use some help with the documentation in the Javadoc in HConnection and HConnectionManager.
The difference between HCM#getConnection and HCM#createConnection is that the latter will ALWAYS create a "cluster connection"? That sounds good.
(How hard to change HTable into an Interface?)
API looks good to me.
the difference between HCM#getConnection and HCM#createConnection is that the latter will ALWAYS create a "cluster connection"?
Yep. It also indicates that the caller needs to manage the lifecycle (i.e. eventually close()'ing the connection).
Should probably deprecate HCM#getConnection as well.
(How hard to change HTable into an Interface?)
HTable is the implementation of HTableInterface. The patch only exposes HTableInterface.
Should we rename HTableInterface to HTable and HTable to HTableImplementation?
Oh, I see. You saying HTable should not have any public constructors? Can deprecate those as well.
Also, do this in 0.94? So we get the deprecation in?
I'm fine with punting on this in 0.94 as it is really just convenience. The existing HTable constructors provide all this functionality (albeit in a more clumsy way).
Could do it in 0.94. Doing the deprecation this late in the game probably doesn't really count toward 0.96.
+1 to deprecate it in 0.94 & 0.96, keeping it in 0.96, removing it in 0.98.
Cool. I'll remove all the old cruft in a separate trunk only jira: HTablePool, HCM#getConnection and all the related caching and cache key generation, the public constructors on HTable, etc.
(We'll be spinning 0.96 from the 0.95 branch, right?)
+1 on patch and deprecation plan as described by nkeywal. Looks like the patch has tabs in it though?
I'll work on some test changes to use the new APIs, and spend some time on the JavaDoc in HCM.
Going to commit to all branches (including 0.94) early next week.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12595554/6580-trunk.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
-1 javadoc. The javadoc tool appears to have generated 4 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 lineLengths. The patch does not introduce lines longer than 100
+1 site. The mvn site goal succeeds with this patch.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestClientNoCluster
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6592//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6592//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6592//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6592//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6592//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6592//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6592//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6592//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6592//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6592//console
This message is automatically generated.
Updated patch. Previous version did not make the required constructor accessible.
Also modified Javadoc some and added basic section to book.xml.
(I expect the JavaDoc/book.xml updated further in trunk when all the connection caching nonsense is removed)
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12595834/6580-trunk-v2.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
-1 javadoc. The javadoc tool appears to have generated 4 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 lineLengths. The patch does not introduce lines longer than 100
+1 site. The mvn site goal succeeds with this patch.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestClientNoCluster
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6600//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6600//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6600//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6600//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6600//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6600//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6600//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6600//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6600//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6600//console
This message is automatically generated.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12595845/6580-trunk-v3.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 9 new or modified tests.
+1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
-1 javadoc. The javadoc tool appears to have generated 4 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 lineLengths. The patch does not introduce lines longer than 100
+1 site. The mvn site goal succeeds with this patch.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor
org.apache.hadoop.hbase.client.TestFromClientSide
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6603//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6603//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6603//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6603//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6603//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6603//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6603//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6603//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6603//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6603//console
This message is automatically generated.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12595855/6580-trunk-v4.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 9 new or modified tests.
+1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
-1 javadoc. The javadoc tool appears to have generated 4 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 lineLengths. The patch does not introduce lines longer than 100
+1 site. The mvn site goal succeeds with this patch.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.zookeeper.lock.TestZKInterProcessReadWriteLock
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6604//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6604//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6604//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6604//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6604//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6604//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6604//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6604//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6604//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6604//console
This message is automatically generated.
HBASE-7980 TestZKInterProcessReadWriteLock is known flakey (I should just disable it since no one is looking at it).
Patch lgtm on skim. Hurray HTablePool is deprecated!
Added getTable(String, ExecutorService) API. Added simple test to TestHCM to exercise new APIs.
Should be getting close.
I might want to move this to 0.94.12, so that we can do proper review/testing on this (I would like to spin 0.94.11 today).
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12596185/6580-trunk-v5.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 9 new or modified tests.
+1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
-1 javadoc. The javadoc tool appears to have generated 4 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 lineLengths. The patch does not introduce lines longer than 100
+1 site. The mvn site goal succeeds with this patch.
+1 core tests. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6609//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6609//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6609//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6609//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6609//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6609//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6609//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6609//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6609//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6609//console
This message is automatically generated.
0.94 test results:
Tests run: 1377, Failures: 0, Errors: 0, Skipped: 13
Please have a closer look. I am planning to commit this to all branches soon.
(I'll also fix the javadoc warning before commit)
+1
nit, in several places:
+ * be create for each using thread.
'create' -> 'created'
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12596665/6580-trunk-v6.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 9 new or modified tests.
+1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 lineLengths. The patch does not introduce lines longer than 100
+1 site. The mvn site goal succeeds with this patch.
+1 core tests. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6638//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6638//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6638//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6638//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6638//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6638//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6638//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6638//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6638//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6638//console
This message is automatically generated.
SUCCESS: Integrated in HBase-0.94 #1099 (See https://builds.apache.org/job/HBase-0.94/1099/)
HBASE-6580 Deprecate HTablePool in favor of HConnection.getTable(...) (larsh: rev 1511545)
- /hbase/branches/0.94/src/docbkx/book.xml
- /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HConnection.java
- /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
- /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HTablePool.java
- /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java
- /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
- /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
- /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestHConnection.java
SUCCESS: Integrated in HBase-0.94-security #248 (See https://builds.apache.org/job/HBase-0.94-security/248/)
HBASE-6580 Deprecate HTablePool in favor of HConnection.getTable(...) (larsh: rev 1511545)
- /hbase/branches/0.94/src/docbkx/book.xml
- /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HConnection.java
- /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
- /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HTablePool.java
- /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java
- /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
- /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
- /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestHConnection.java
SUCCESS: Integrated in hbase-0.95 #414 (See https://builds.apache.org/job/hbase-0.95/414/)
HBASE-6580 Deprecate HTablePool in favor of HConnection.getTable(...) (larsh: rev 1511544)
- /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnection.java
- /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
- /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionWrapper.java
- /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTablePool.java
- /hbase/branches/0.95/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestClientNoCluster.java
- /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
- /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
- /hbase/branches/0.95/src/main/docbkx/book.xml
SUCCESS: Integrated in HBase-TRUNK #4353 (See https://builds.apache.org/job/HBase-TRUNK/4353/)
HBASE-6580 Deprecate HTablePool in favor of HConnection.getTable(...) (larsh: rev 1511543)
- /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnection.java
- /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
- /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionWrapper.java
- /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTablePool.java
- /hbase/trunk/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestClientNoCluster.java
- /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
- /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
- /hbase/trunk/src/main/docbkx/book.xml
FAILURE: Integrated in hbase-0.95-on-hadoop2 #224 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/224/)
HBASE-6580 Deprecate HTablePool in favor of HConnection.getTable(...) (larsh: rev 1511544)
- /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnection.java
- /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
- /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionWrapper.java
- /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTablePool.java
- /hbase/branches/0.95/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestClientNoCluster.java
- /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
- /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
- /hbase/branches/0.95/src/main/docbkx/book.xml
FAILURE: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #657 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/657/)
HBASE-6580 Deprecate HTablePool in favor of HConnection.getTable(...) (larsh: rev 1511543)
- /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnection.java
- /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
- /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionWrapper.java
- /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTablePool.java
- /hbase/trunk/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestClientNoCluster.java
- /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
- /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
- /hbase/trunk/src/main/docbkx/book.xml
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #2873 (See https://builds.apache.org/job/HBase-Trunk_matrix/2873/)
HBASE-17903 Corrected the alias for the link of HBASE-6580 (chia7712: rev 918aa4655c4109159f27b6d78460bd3681c11f06)
- (edit) src/main/asciidoc/_chapters/architecture.adoc
ABORTED: Integrated in Jenkins build HBase-HBASE-14614 #190 (See https://builds.apache.org/job/HBase-HBASE-14614/190/)
HBASE-17903 Corrected the alias for the link of HBASE-6580 (chia7712: rev 918aa4655c4109159f27b6d78460bd3681c11f06)
- (edit) src/main/asciidoc/_chapters/architecture.adoc
HConnectionPool ?
Edit: NM, that's not right either.