Details
Description
Hi,
When compiling with
./make-distribution.sh --hadoop 2.0.0-cdh4.6.0 --with-yarn --tgz
I have the following errors on streaming java api:
Version is 0.9.0-incubating
Making spark-0.9.0-incubating-hadoop_2.0.0-cdh4.6.0-bin.tar.gz
Hadoop version set to 2.0.0-cdh4.6.0
YARN enabled
[info] Loading project definition from /root/spark-source/project/project
[info] Loading project definition from /root/spark-source/project
[info] Set current project to root (in build file:/root/spark-source/)
[info] Compiling 1 Scala source to /root/spark-source/streaming/target/scala-2.10/classes...
[error] /root/spark-source/streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaPairDStream.scala:57: type mismatch;
[error] found : org.apache.spark.streaming.dstream.DStream[(K, V)]
[error] required: org.apache.spark.streaming.api.java.JavaPairDStream[K,V]
[error] Note: implicit method fromPairDStream is not applicable here because it comes after the application point and it lacks an explicit result type
[error] dstream.filter((x => f.booleanValue()))
[error] ^
[error] /root/spark-source/streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaPairDStream.scala:60: type mismatch;
[error] found : org.apache.spark.streaming.dstream.DStream[(K, V)]
[error] required: org.apache.spark.streaming.api.java.JavaPairDStream[K,V]
[error] Note: implicit method fromPairDStream is not applicable here because it comes after the application point and it lacks an explicit result type
[error] def cache(): JavaPairDStream[K, V] = dstream.cache()
[error] ^
[error] /root/spark-source/streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaPairDStream.scala:63: type mismatch;
[error] found : org.apache.spark.streaming.dstream.DStream[(K, V)]
[error] required: org.apache.spark.streaming.api.java.JavaPairDStream[K,V]
[error] Note: implicit method fromPairDStream is not applicable here because it comes after the application point and it lacks an explicit result type
[error] def persist(): JavaPairDStream[K, V] = dstream.persist()
[error] ^......
[error] /root/spark-source/streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaPairDStream.scala:669: type mismatch;
[error] found : org.apache.spark.streaming.dstream.DStream[(K, (com.google.common.base.Optional[V], W))]
[error] required: org.apache.spark.streaming.api.java.JavaPairDStream[K,(com.google.common.base.Optional[V], W)]
[error] Note: implicit method fromPairDStream is not applicable here because it comes after the application point and it lacks an explicit result type
[error] joinResult.mapValuesUnknown macro: {case (v, w) => (JavaUtils.optionToOptional(v), w)}[error] ^
[error] 44 errors found
[error] (streaming/compile:compile) Compilation failed
Here is a simple PR fix this problem: https://github.com/apache/spark/pull/153
Attachments
Issue Links
- links to