Description
I built Mahout 0.9 with Bigtop and found it couldn't work with hadoop 2.x.
Exception in thread "main" java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected at org.apache.mahout.common.HadoopUtil.getCustomJobName(HadoopUtil.java:174) at org.apache.mahout.common.AbstractJob.prepareJob(AbstractJob.java:614) at org.apache.mahout.cf.taste.hadoop.preparation.PreparePreferenceMatrixJob.run(PreparePreferenceMatrixJob.java:73) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.run(RecommenderJob.java:164) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.main(RecommenderJob.java:322) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72) at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144) at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:152) at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
The root cause is that bigtop uses the wrong parameter hadoop2.version when building Mahout.
mvn clean install -Dmahout.skip.distribution=false -DskipTests -Dhadoop2.version=$HADOOP_VERSION "$@"
For Mahout 0.9 , the correct property is hadoop.version in pom.xml of Mahout 0.9
<hadoop.version>1.2.1</hadoop.version>
Attachments
Issue Links
- is related to
-
MAHOUT-1329 Mahout for hadoop 2
- Closed
- relates to
-
MAHOUT-1329 Mahout for hadoop 2
- Closed
1.
|
0.8 RC: Looks like it is building Mahout1x ? | Resolved | Unassigned |