Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-880

Program with two BroadcastVars causes CompilerException

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • pre-apache
    • None

    Description

      The following CompilerException is raised by

      ```
      eu.stratosphere.compiler.CompilerException:
      No plan meeting the requirements could be created @ GroupReduce (eu.stratosphere.example.java.clustering.Prog$IdentityReduce) (1:null).
      Most likely reason: Too restrictive plan hints.
      ```

      this program.

      ```java
      public class Prog {

      @SuppressWarnings("unchecked")
      public static void main(String[] args) throws Exception

      { final ExecutionEnvironment env = ExecutionEnvironment .getExecutionEnvironment(); DataSet<Integer> bc1 = env.fromElements(0); DataSet<Integer> bc2 = env.fromElements(0); DataSet<Tuple1<Integer>> data = env.fromElements( new Tuple1<Integer>(0), new Tuple1<Integer>(0)); DataSet<Tuple1<Integer>> data2 = data .flatMap(new IdentityMap()) .withBroadcastSet(bc1, "BC1"); DataSet<Tuple1<Integer>> data3 = data2 .groupBy(0) .reduceGroup(new IdentityReduce()) .withBroadcastSet(bc1, "BC1") .withBroadcastSet(bc2, "BC2"); data3 .flatMap(new IdentityMap()) .print(); data2 .flatMap(new IdentityMap()) .print(); env.setDegreeOfParallelism(1); env.execute("CustomALS"); }

      public static final class IdentityMap extends
      FlatMapFunction<Tuple1<Integer>, Tuple1<Integer>> {
      @Override
      public void flatMap(Tuple1<Integer> value,
      Collector<Tuple1<Integer>> out)

      { out.collect(value); }

      }

      public static final class IdentityReduce extends
      GroupReduceFunction<Tuple1<Integer>, Tuple1<Integer>> {
      @Override
      public void reduce(Iterator<Tuple1<Integer>> values,
      Collector<Tuple1<Integer>> out) {
      while (values.hasNext())

      { out.collect(values.next()); }

      }
      }
      }

      ```

      ---------------- Imported from GitHub ----------------
      Url: https://github.com/stratosphere/stratosphere/issues/880
      Created by: fhueske
      Labels: bug, optimizer,
      Milestone: Release 0.5.1
      Created at: Wed May 28 15:44:15 CEST 2014
      State: open

      Attachments

        Activity

          People

            Unassigned Unassigned
            github-import GitHub Import
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: