Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-4473

NPE when executors are down but events are still coming in

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 0.90.4
    • 0.90.5
    • None
    • None
    • Reviewed

    Description

      Minor annoyance when shutting down a cluster and the master is still receiving events from Zookeeper:

      2011-09-22 23:53:01,552 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher: master:60000-0x3292d87deb004f Received InterruptedException, doing nothing here
      java.lang.InterruptedException
      at java.lang.Object.wait(Native Method)
      at java.lang.Object.wait(Object.java:485)
      at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1317)
      at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:726)
      at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:938)
      at org.apache.hadoop.hbase.zookeeper.ZKAssign.deleteNode(ZKAssign.java:407)
      at org.apache.hadoop.hbase.zookeeper.ZKAssign.deleteOpenedNode(ZKAssign.java:284)
      at org.apache.hadoop.hbase.master.handler.OpenedRegionHandler.process(OpenedRegionHandler.java:88)
      at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:156)
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      at java.lang.Thread.run(Thread.java:662)
      ...
      2011-09-22 23:53:01,558 DEBUG org.apache.hadoop.hbase.executor.ExecutorService: Executor service [MASTER_OPEN_REGION-sv2borg170:60000] not found in {}
      2011-09-22 23:53:01,558 ERROR org.apache.zookeeper.ClientCnxn: Error while calling watcher
      java.lang.NullPointerException
      at org.apache.hadoop.hbase.executor.ExecutorService.submit(ExecutorService.java:220)
      at org.apache.hadoop.hbase.master.AssignmentManager.handleRegion(AssignmentManager.java:447)
      at org.apache.hadoop.hbase.master.AssignmentManager.nodeDataChanged(AssignmentManager.java:546)
      at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:281)
      at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
      at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)

      It's annoying because it then spams you with a bunch of NPEs that have nothing to do with the reason the Master is shutting down. Googling I saw someone also had that issue in June: http://pastebin.com/5Tqrj0nq

      Attachments

        1. HBASE-4473-0.90.patch
          2 kB
          Jean-Daniel Cryans
        2. HBASE-4473.patch
          2 kB
          Jean-Daniel Cryans
        3. HBASE-4473-v2.patch
          2 kB
          Jean-Daniel Cryans

        Activity

          People

            jdcryans Jean-Daniel Cryans
            jdcryans Jean-Daniel Cryans
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: