Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-19246

TableSourceITCase.testStreamScanParallelism fails on private Azure accounts

    XMLWordPrintableJSON

Details

    Description

      Example: https://dev.azure.com/rmetzger/Flink/_build/results?buildId=8381&view=logs&j=69332ead-8935-5abf-5b3d-e4280fb1ff4c&t=58eb9526-6bcb-5835-ae76-f5bd5f6df6ac
      or
      https://dev.azure.com/rmetzger/Flink/_build/results?buildId=8379&view=logs&j=69332ead-8935-5abf-5b3d-e4280fb1ff4c&t=58eb9526-6bcb-5835-ae76-f5bd5f6df6ac
      or
      https://dev.azure.com/rmetzger/Flink/_build/results?buildId=8369&view=logs&j=69332ead-8935-5abf-5b3d-e4280fb1ff4c&t=58eb9526-6bcb-5835-ae76-f5bd5f6df6ac
      (this change is already merged to master, so it is unlikely to cause the error)

      2020-09-15T13:51:34.6773312Z org.apache.flink.api.common.InvalidProgramException: The implementation of the CollectionInputFormat is not serializable. The object probably contains or references non serializable fields.
      2020-09-15T13:51:34.6774140Z 	at org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:151)
      2020-09-15T13:51:34.6774634Z 	at org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:126)
      2020-09-15T13:51:34.6775136Z 	at org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:71)
      2020-09-15T13:51:34.6775728Z 	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.clean(StreamExecutionEnvironment.java:1913)
      2020-09-15T13:51:34.6776617Z 	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.java:1601)
      2020-09-15T13:51:34.6777322Z 	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.createInput(StreamExecutionEnvironment.java:1493)
      2020-09-15T13:51:34.6778029Z 	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.createInput(StreamExecutionEnvironment.java:1483)
      2020-09-15T13:51:34.6778887Z 	at org.apache.flink.table.factories.utils.TestCollectionTableFactory$CollectionTableSource.getDataStream(TestCollectionTableFactory.scala:159)
      2020-09-15T13:51:34.6779659Z 	at org.apache.flink.table.factories.utils.TestCollectionTableFactory$CollectionTableSource.getDataStream(TestCollectionTableFactory.scala:134)
      2020-09-15T13:51:34.6780394Z 	at org.apache.flink.table.plan.nodes.datastream.StreamTableSourceScan.translateToPlan(StreamTableSourceScan.scala:105)
      2020-09-15T13:51:34.6781058Z 	at org.apache.flink.table.plan.nodes.datastream.DataStreamSink.translateInput(DataStreamSink.scala:189)
      2020-09-15T13:51:34.6781655Z 	at org.apache.flink.table.plan.nodes.datastream.DataStreamSink.writeToSink(DataStreamSink.scala:84)
      2020-09-15T13:51:34.6782266Z 	at org.apache.flink.table.plan.nodes.datastream.DataStreamSink.translateToPlan(DataStreamSink.scala:59)
      2020-09-15T13:51:34.6782951Z 	at org.apache.flink.table.planner.StreamPlanner.org$apache$flink$table$planner$StreamPlanner$$translateToCRow(StreamPlanner.scala:274)
      2020-09-15T13:51:34.6783640Z 	at org.apache.flink.table.planner.StreamPlanner$$anonfun$translate$1.apply(StreamPlanner.scala:119)
      2020-09-15T13:51:34.6784227Z 	at org.apache.flink.table.planner.StreamPlanner$$anonfun$translate$1.apply(StreamPlanner.scala:116)
      2020-09-15T13:51:34.6784799Z 	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
      2020-09-15T13:51:34.6785345Z 	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
      2020-09-15T13:51:34.6785828Z 	at scala.collection.Iterator$class.foreach(Iterator.scala:891)
      2020-09-15T13:51:34.6786285Z 	at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
      2020-09-15T13:51:34.6786760Z 	at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
      2020-09-15T13:51:34.6787210Z 	at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
      2020-09-15T13:51:34.6787681Z 	at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
      2020-09-15T13:51:34.6788168Z 	at scala.collection.AbstractTraversable.map(Traversable.scala:104)
      2020-09-15T13:51:34.6788648Z 	at org.apache.flink.table.planner.StreamPlanner.translate(StreamPlanner.scala:116)
      2020-09-15T13:51:34.6789286Z 	at org.apache.flink.table.api.bridge.scala.internal.StreamTableEnvironmentImpl.toDataStream(StreamTableEnvironmentImpl.scala:178)
      2020-09-15T13:51:34.6790031Z 	at org.apache.flink.table.api.bridge.scala.internal.StreamTableEnvironmentImpl.toAppendStream(StreamTableEnvironmentImpl.scala:103)
      2020-09-15T13:51:34.6790705Z 	at org.apache.flink.table.api.bridge.scala.TableConversions.toAppendStream(TableConversions.scala:78)
      2020-09-15T13:51:34.6791362Z 	at org.apache.flink.table.runtime.stream.table.TableSourceITCase.testStreamScanParallelism(TableSourceITCase.scala:118)
      2020-09-15T13:51:34.6791907Z 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      2020-09-15T13:51:34.6792350Z 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      2020-09-15T13:51:34.6792906Z 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      2020-09-15T13:51:34.6793391Z 	at java.lang.reflect.Method.invoke(Method.java:498)
      2020-09-15T13:51:34.6793869Z 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
      2020-09-15T13:51:34.6794406Z 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      2020-09-15T13:51:34.6795396Z 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
      2020-09-15T13:51:34.6796052Z 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      2020-09-15T13:51:34.6796565Z 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
      2020-09-15T13:51:34.6797051Z 	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
      2020-09-15T13:51:34.6797481Z 	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
      2020-09-15T13:51:34.6797896Z 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
      2020-09-15T13:51:34.6798465Z 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
      2020-09-15T13:51:34.6799768Z 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
      2020-09-15T13:51:34.6800273Z 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
      2020-09-15T13:51:34.6800728Z 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
      2020-09-15T13:51:34.6801197Z 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
      2020-09-15T13:51:34.6801656Z 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
      2020-09-15T13:51:34.6802119Z 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
      2020-09-15T13:51:34.6802599Z 	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
      2020-09-15T13:51:34.6803183Z 	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
      2020-09-15T13:51:34.6803687Z 	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
      2020-09-15T13:51:34.6804117Z 	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
      2020-09-15T13:51:34.6804589Z 	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
      2020-09-15T13:51:34.6805148Z 	at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
      2020-09-15T13:51:34.6805721Z 	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
      2020-09-15T13:51:34.6806253Z 	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
      2020-09-15T13:51:34.6806844Z 	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
      2020-09-15T13:51:34.6807445Z 	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
      2020-09-15T13:51:34.6807972Z 	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
      2020-09-15T13:51:34.6808487Z 	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
      2020-09-15T13:51:34.6809041Z Caused by: java.lang.RuntimeException: Row arity of record (3) does not match this serializers field length (1).
      2020-09-15T13:51:34.6809610Z 	at org.apache.flink.api.java.typeutils.runtime.RowSerializer.serialize(RowSerializer.java:181)
      2020-09-15T13:51:34.6810199Z 	at org.apache.flink.api.java.typeutils.runtime.RowSerializer.serialize(RowSerializer.java:58)
      2020-09-15T13:51:34.6810784Z 	at org.apache.flink.api.java.io.CollectionInputFormat.writeObject(CollectionInputFormat.java:90)
      2020-09-15T13:51:34.6811287Z 	at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
      2020-09-15T13:51:34.6811746Z 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      2020-09-15T13:51:34.6814374Z 	at java.lang.reflect.Method.invoke(Method.java:498)
      2020-09-15T13:51:34.6814842Z 	at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1140)
      2020-09-15T13:51:34.6815329Z 	at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496)
      2020-09-15T13:51:34.6815850Z 	at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
      2020-09-15T13:51:34.6816355Z 	at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
      2020-09-15T13:51:34.6816826Z 	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
      2020-09-15T13:51:34.6817345Z 	at org.apache.flink.util.InstantiationUtil.serializeObject(InstantiationUtil.java:586)
      2020-09-15T13:51:34.6817871Z 	at org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:133)
      2020-09-15T13:51:34.6818350Z 	... 59 more
      

      I don't understand why this is failing only in my personal azure account (on branches with different changes).

      Attachments

        Issue Links

          Activity

            People

              jark Jark Wu
              rmetzger Robert Metzger
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: