Major Description
Release connection to pool after HTTP call
The close() method in BasicResponse does only EntityUtils.consumeQuietly(response.getEntity()):
public void close() { this.consume(); } public void consume() { if(!this.consumed) { EntityUtils.consumeQuietly(this.response.getEntity()); this.consumed = true; } }
The underlying HTTP connection is still held. In order to ensure correct deallocation of OS resources we need to call CloseableHttpResponse#close() like this:
public void close() { try { this.consume(); } finally { if (response instanceof CloseableHttpResponse) { try { ((CloseableHttpResponse) response).close(); } catch (IOException e) { throw Throwables.propagate(e); } } } }
Shutdown connection pool
Also when the Hadoop session is no more used, the only method present by to be call is shutdown(), but it close only the ExecutorService. When a CloseableHttpClient instance is no longer needed, we have to shut down the connection manager to ensure immediate deallocation of all OS resources.
public void close() { try { executor.shutdownNow(); } catch(Exception e) { // log something here } try { client.close(); // client.close() should call getConnectionManager().shutdown(); } catch(Exception e) { // log something here } }