Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
1.2.0
-
None
Description
PR https://github.com/apache/spark/pull/1813 shaded Guava jar file and moved Guava classes to package org.spark-project.guava when Spark is built by Maven.
When a user jar uses the actual com.google.common.base.Optional transform(com.google.common.base.Function); method from Guava, a java.lang.NoSuchMethodError: com.google.common.base.Optional.transform(Lcom/google/common/base/Function;)Lcom/google/common/base/Optional; is thrown.
The reason seems to be that the Optional class included on spark-assembly-1.2.0-hadoop1.0.4.jar has an incorrect method signature that includes the shaded class as an argument:
Expected:
javap -classpath target/scala-2.10/googlegenomics-spark-examples-assembly-1.0.jar com.google.common.base.Optional
public abstract <V extends java/lang/Object> com.google.common.base.Optional<V> transform(com.google.common.base.Function<? super T, V>);
Found:
javap -classpath lib/spark-assembly-1.2.0-hadoop1.0.4.jar com.google.common.base.Optional
public abstract <V extends java/lang/Object> com.google.common.base.Optional<V> transform(org.spark-project.guava.common.base.Function<? super T, V>);
Attachments
Issue Links
- is related to
-
SPARK-2848 Shade Guava in Spark deliverables
- Resolved
- links to