Description
Root cause is OOME on the region server. Nonetheless a hangup during IPC causes the client to fail the write, currently causing data loss. Should the application catch and retry? Or should the client libraries try harder?
Dec 4, 2008 5:25:30 PM com.powerset.heritrix.writer.HBaseWriterProcessor innerProcessResult
SEVERE: Failed write of Record: http://www.publicrecordslocal.com/georgia.htm (in thread 'ToeThread #9: http://www.publicrecordslocal.com/georgia.htm'; in processor 'Archiver')
java.io.IOException: java.io.IOException: Call to /10.30.94.38:60020 failed on local exception: Connection refused
at com.powerset.heritrix.writer.HBaseWriter.write(Unknown Source)
at com.powerset.heritrix.writer.HBaseWriterProcessor.write(Unknown Source)
at com.powerset.heritrix.writer.HBaseWriterProcessor.innerProcessResult(Unknown Source)
at org.archive.modules.Processor.process(Processor.java:123)
at org.archive.crawler.framework.ToeThread.processCrawlUri(ToeThread.java:310)
at org.archive.crawler.framework.ToeThread.run(ToeThread.java:157)
Caused by: java.io.IOException: Call to /10.30.94.38:60020 failed on local exception: Connection refused
at org.apache.hadoop.ipc.Client.call(Client.java:699)
at org.apache.hadoop.hbase.ipc.HbaseRPC$Invoker.invoke(HbaseRPC.java:323)
at $Proxy12.batchUpdates(Unknown Source)
at org.apache.hadoop.hbase.client.HConnectionManager$TableServers$2.call(HConnectionManager.java:919)
at org.apache.hadoop.hbase.client.HConnectionManager$TableServers$2.call(HConnectionManager.java:917)
at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerForWithoutRetries(HConnectionManager.java:875)
at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.processBatchOfRows(HConnectionManager.java:916)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1267)
at org.apache.hadoop.hbase.client.HTable.commit(HTable.java:1238)
at org.apache.hadoop.hbase.client.HTable.commit(HTable.java:1218)
at net.iridiant.content.Content.storeURLInfo(Unknown Source)
... 6 more
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:299)
at org.apache.hadoop.ipc.Client$Connection.access$1700(Client.java:176)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:772)
at org.apache.hadoop.ipc.Client.call(Client.java:685)
... 16 more
Attachments
Attachments
Issue Links
- is related to
-
HBASE-1040 OOME does not cause graceful shutdown under some failure scenarios
- Closed
-
HBASE-1038 Master should accept client feedback about inconsistencies in META
- Closed