[S4-131] checkpoint error "CancellationException" - ASF JIRA

Details

Type: Bug
Status: Resolved
Priority: Minor
Resolution: Fixed
Affects Version/s: 0.6
Fix Version/s: 0.7
Labels:
None
Environment:

Hide

The configure like this:
App program:
...
wordSumPE.setCheckpointingConfig(new CheckpointingConfig.Builder(CheckpointingMode.TIME).frequency(20).timeUnit(TimeUnit.SECONDS).build());
...
delpoy:
./s4 deploy -a=example.wordcountApp -c=testCluster1 -appName=wordApp -p=s4.checkpointing.filesystem.storageRootPath=/home/tmp/s4checkpoint -emc=org.apache.s4.core.ft.FileSystemBackendCheckpointingModule

Show
The configure like this: App program: ... wordSumPE.setCheckpointingConfig(new CheckpointingConfig.Builder(CheckpointingMode.TIME).frequency(20).timeUnit(TimeUnit.SECONDS).build()); ... delpoy: ./s4 deploy -a=example.wordcountApp -c=testCluster1 -appName=wordApp -p=s4.checkpointing.filesystem.storageRootPath=/home/tmp/s4checkpoint -emc=org.apache.s4.core.ft.FileSystemBackendCheckpointingModule

Description

Then I get this error:
14:21:50.251 [Checkpointing-storage-0] WARN org.apache.s4.core.ft.SaveStateTask - Cannot save checkpoint : [PROTO_ID];[KEY] --> [example.WordSumPE];[word]
java.util.concurrent.ExecutionException: com.esotericsoftware.kryo.KryoException: java.util.ConcurrentModificationException
Serialization trace:
classes (sun.misc.Launcher$AppClassLoader)
contextClassLoader (java.lang.Thread)
thread (java.util.concurrent.ThreadPoolExecutor$Worker)
workers (java.util.concurrent.ThreadPoolExecutor)
fetchingThreadPool (org.apache.s4.core.ft.SafeKeeper)
checkpointingFramework (example.wordcountApp)
app (org.apache.s4.core.Stream)
downStream (example.WordSumPE)
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232) ~[na:1.6.0_22]
at java.util.concurrent.FutureTask.get(FutureTask.java:91) ~[na:1.6.0_22]
at org.apache.s4.core.ft.SaveStateTask.run(SaveStateTask.java:66) ~[bin/:na]
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [na:1.6.0_22]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [na:1.6.0_22]
at java.lang.Thread.run(Thread.java:662) [na:1.6.0_22]
Caused by: com.esotericsoftware.kryo.KryoException: java.util.ConcurrentModificationException
Serialization trace:
classes (sun.misc.Launcher$AppClassLoader)
contextClassLoader (java.lang.Thread)
thread (java.util.concurrent.ThreadPoolExecutor$Worker)
workers (java.util.concurrent.ThreadPoolExecutor)
fetchingThreadPool (org.apache.s4.core.ft.SafeKeeper)
checkpointingFramework (example.wordcountApp)
app (org.apache.s4.core.Stream)
downStream (example.WordSumPE)
at com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:585) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:504) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:564) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:504) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:564) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.Kryo.writeObjectOrNull(Kryo.java:552) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:68) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:18) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:504) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:564) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:504) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:564) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:504) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:564) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:504) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:564) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:504) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:564) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:571) ~[kryo-2.20.jar:na]
at org.apache.s4.comm.serialize.KryoSerDeser.serialize(KryoSerDeser.java:91) ~[bin/:na]
at org.apache.s4.core.ProcessingElement.serializeState(ProcessingElement.java:802) ~[bin/:na]
at org.apache.s4.core.ft.SerializeTask.call(SerializeTask.java:42) ~[bin/:na]
at org.apache.s4.core.ft.SerializeTask.call(SerializeTask.java:1) ~[bin/:na]
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) ~[na:1.6.0_22]
at java.util.concurrent.FutureTask.run(FutureTask.java:138) ~[na:1.6.0_22]
... 3 common frames omitted
Caused by: java.util.ConcurrentModificationException: null
at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372) ~[na:1.6.0_22]
at java.util.AbstractList$Itr.next(AbstractList.java:343) ~[na:1.6.0_22]
at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:74) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:18) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:504) ~[kryo-2.20.jar:na]
at com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:564) ~[kryo-2.20.jar:na]
... 35 common frames omitted

I debug the program and at the position : SaveStateTask.run "futureSerializedState.get(1000, TimeUnit.MILLISECONDS)." The futureTask get "CancellationException" displayed above error.

checkpoint error "CancellationException"

Details

Description

Attachments

Activity

People

Dates