Major Description
We are receiving the following exception when using Pig
Caused by: java.lang.ClassCastException: org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject cannot be cast to org.apache.pig.backend.hadoop.executionen\
gine.physicalLayer.expressionOperators.PORelationToExprProject
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.PORelationToExprProject.clone(PORelationToExprProject.java:144)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.PORelationToExprProject.clone(PORelationToExprProject.java:50)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.plans.PhysicalPlan.clone(PhysicalPlan.java:227)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.clone(POForEach.java:639)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.clone(POForEach.java:53)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.plans.PhysicalPlan.clone(PhysicalPlan.java:227)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer.mergeDiamondMROper(MultiQueryOptimizer.java:298)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer.visitMROp(MultiQueryOptimizer.java:219)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:273)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:46)
at org.apache.pig.impl.plan.ReverseDependencyOrderWalker.walk(ReverseDependencyOrderWalker.java:71)
at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:46)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer.visit(MultiQueryOptimizer.java:94)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(MapReduceLauncher.java:629)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:148)
at org.apache.pig.PigServer.launchPlan(PigServer.java:1264)
On further investigation it seems that POProject's clone method is implemented as
@Override
public POProject clone() throws CloneNotSupportedException {
ArrayList<Integer> cols = new ArrayList<Integer>(columns.size());
// Can resuse the same Integer objects, as they are immutable
for (Integer i : columns) {
cols.add(i);
}
POProject clone = new POProject(new OperatorKey(mKey.scope,
NodeIdGenerator.getGenerator().getNextNodeId(mKey.scope)),
requestedParallelism, cols);
clone.cloneHelper(this);
clone.overloaded = overloaded;
clone.startCol = startCol;
clone.isProjectToEnd = isProjectToEnd;
clone.resultType = resultType;
return clone;
}
It uses a constructor to clone POProject (which break the weak rule of object cloning)
In the subclass , PORelationToExprProject implements cloneable as
@Override
public PORelationToExprProject clone() throws CloneNotSupportedException {
return (PORelationToExprProject) super.clone();
}
As seen from the POProject's implementation of cloneable, super.clone will never be of type PORelationToExprProject,