Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-4752

Hadoop examples fail for hadoop accelerator

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 1.9
    • 1.9
    • None
    • None

    Description

      0. Setup Apache Ignite Hadoop accelerator
      1. Start 4 nodes bin/ignite.sh
      2. Start a hadoop example, for instance pi
      3. The server nodes print out following:

      [23:51:47,364][ERROR][pub-#62%null%][HadoopJobTracker] Failed to submit job: f05d867c-a96b-4ea4-be57-0a6002f14870_1
      class org.apache.ignite.IgniteCheckedException: null
      	at org.apache.ignite.internal.processors.hadoop.HadoopDefaultJobInfo.createJob(HadoopDefaultJobInfo.java:99)
      	at org.apache.ignite.internal.processors.hadoop.jobtracker.HadoopJobTracker.job(HadoopJobTracker.java:1120)
      	at org.apache.ignite.internal.processors.hadoop.jobtracker.HadoopJobTracker.submit(HadoopJobTracker.java:313)
      	at org.apache.ignite.internal.processors.hadoop.HadoopProcessor.submit(HadoopProcessor.java:173)
      	at org.apache.ignite.internal.processors.hadoop.HadoopImpl.submit(HadoopImpl.java:69)
      	at org.apache.ignite.internal.processors.hadoop.proto.HadoopProtocolSubmitJobTask.run(HadoopProtocolSubmitJobTask.java:50)
      	at org.apache.ignite.internal.processors.hadoop.proto.HadoopProtocolSubmitJobTask.run(HadoopProtocolSubmitJobTask.java:33)
      	at org.apache.ignite.internal.processors.hadoop.proto.HadoopProtocolTaskAdapter$Job.execute(HadoopProtocolTaskAdapter.java:101)
      	at org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:560)
      	at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6618)
      	at org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:554)
      	at org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:483)
      	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
      	at org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1180)
      	at org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1894)
      	at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1216)
      	at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:844)
      	at org.apache.ignite.internal.managers.communication.GridIoManager.access$2100(GridIoManager.java:108)
      	at org.apache.ignite.internal.managers.communication.GridIoManager$6.run(GridIoManager.java:783)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: java.lang.reflect.InvocationTargetException
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
      	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
      	at org.apache.ignite.internal.processors.hadoop.HadoopDefaultJobInfo.createJob(HadoopDefaultJobInfo.java:93)
      	... 21 more
      Caused by: java.lang.NoClassDefFoundError: com/google/common/base/Preconditions
      	at org.apache.hadoop.conf.Configuration$DeprecationDelta.<init>(Configuration.java:314)
      	at org.apache.hadoop.conf.Configuration$DeprecationDelta.<init>(Configuration.java:327)
      	at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:409)
      	at org.apache.ignite.internal.processors.hadoop.impl.v2.HadoopV2Job.<init>(HadoopV2Job.java:158)
      	... 26 more
      Caused by: java.lang.ClassNotFoundException: com.google.common.base.Preconditions
      	at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
      	at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
      	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
      	... 30 more
      [23:51:49,788][ERROR][pub-#11%null%][HadoopJobTracker] Failed to submit job: 6e1713a8-8bbf-480c-8e0b-9045bc55c77d_1
      class org.apache.ignite.IgniteCheckedException: null
      	at org.apache.ignite.internal.processors.hadoop.HadoopDefaultJobInfo.createJob(HadoopDefaultJobInfo.java:99)
      	at org.apache.ignite.internal.processors.hadoop.jobtracker.HadoopJobTracker.job(HadoopJobTracker.java:1120)
      	at org.apache.ignite.internal.processors.hadoop.jobtracker.HadoopJobTracker.submit(HadoopJobTracker.java:313)
      	at org.apache.ignite.internal.processors.hadoop.HadoopProcessor.submit(HadoopProcessor.java:173)
      	at org.apache.ignite.internal.processors.hadoop.HadoopImpl.submit(HadoopImpl.java:69)
      	at org.apache.ignite.internal.processors.hadoop.proto.HadoopProtocolSubmitJobTask.run(HadoopProtocolSubmitJobTask.java:50)
      	at org.apache.ignite.internal.processors.hadoop.proto.HadoopProtocolSubmitJobTask.run(HadoopProtocolSubmitJobTask.java:33)
      	at org.apache.ignite.internal.processors.hadoop.proto.HadoopProtocolTaskAdapter$Job.execute(HadoopProtocolTaskAdapter.java:101)
      	at org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:560)
      	at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6618)
      	at org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:554)
      	at org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:483)
      	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
      	at org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1180)
      	at org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1894)
      	at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1216)
      	at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:844)
      	at org.apache.ignite.internal.managers.communication.GridIoManager.access$2100(GridIoManager.java:108)
      	at org.apache.ignite.internal.managers.communication.GridIoManager$6.run(GridIoManager.java:783)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: java.lang.reflect.InvocationTargetException
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
      	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
      	at org.apache.ignite.internal.processors.hadoop.HadoopDefaultJobInfo.createJob(HadoopDefaultJobInfo.java:93)
      	... 21 more
      Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.mapred.JobConf
      	at org.apache.ignite.internal.processors.hadoop.impl.v2.HadoopV2Job.<init>(HadoopV2Job.java:158)
      	... 26 more
      

      The issue was introduced between 5b94a7d5b14600e951d4bb31b474f9fb1195f703 (it works) and a65889a75b0ba7b3fe74d97df82e2214ff1d9b52 (it does not work)

      Attachments

        Activity

          People

            ezhuravl Evgenii Zhuravlev
            skozlov Sergey Kozlov
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 20m
                20m