Uploaded image for project: 'Giraph (Retired)'
  1. Giraph (Retired)
  2. GIRAPH-351

Fail job early when there is no input

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.0.0
    • None
    • None

    Description

      Right now we get a cryptic error message when we know that there is no input!

      2012-09-28 13:45:20,410 INFO org.apache.giraph.graph.BspServiceMaster: generateInputSplits: Got 0 input splits for 100 workers
      2012-09-28 13:45:20,410 WARN org.apache.giraph.graph.BspServiceMaster: createInputSplits: Number of inputSplits=0 < 100=number of healthy processes, some workers will be not used

      But the job keeps on going and the workers get the exception

      java.lang.IllegalStateException: setup: loadVertices failed due to KeeperException
      at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:628)
      at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:370)
      at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:570)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:632)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
      at org.apache.hadoop.mapred.Child.main(Child.java:171)
      Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /_hadoopBsp/job_201209271814.325_0001/_inputSplitDir
      at org.apache.zookeeper.KeeperException.create(KeeperException.java:102)
      at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
      at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1243)
      at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1271)
      at org.apache.giraph.zk.ZooKeeperExt.getChildrenExt(ZooKeeperExt.java:273)
      at org.apache.giraph.graph.BspServiceWorker.reserveInputSplit(BspServiceWorker.java:225)
      at org.apache.giraph.graph.BspServiceWorker.loadVertices(BspServiceWorker.java:320)
      at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:607)
      ... 5 more

      Attachments

        1. GIRAPH-351.patch
          1.0 kB
          Avery Ching
        2. GIRAPH-351.2.patch
          1 kB
          Avery Ching

        Activity

          People

            aching Avery Ching
            aching Avery Ching
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: