Description
When a user tries to spin up more containers than the max partitions or tasks, the leader process gets killed.
We throw an exception in the TaskNameGrouper for the above scenario and that needs to be handled gracefully by the leader and kill the newly spun containers as opposed bailing out.
Here is the stack trace
2017-05-10 15:13:24.526 [debounce-thread-0] ScheduleAfterDebounceTime [ERROR] OnProcessorChange threw an exception. java.lang.IllegalArgumentException: number of containers 2 is bigger than number of tasks 1 at org.apache.samza.container.grouper.task.GroupByContainerIds.group(GroupByContainerIds.java:68) at org.apache.samza.coordinator.JobModelManager$.readJobModel(JobModelManager.scala:258) at org.apache.samza.coordinator.JobModelManager.readJobModel(JobModelManager.scala) at org.apache.samza.zk.ZkJobCoordinator.generateNewJobModel(ZkJobCoordinator.java:212) at org.apache.samza.zk.ZkJobCoordinator.doOnProcessorChange(ZkJobCoordinator.java:125) at org.apache.samza.zk.ZkJobCoordinator.lambda$onProcessorChange$1(ZkJobCoordinator.java:120) at org.apache.samza.zk.ScheduleAfterDebounceTime.lambda$scheduleAfterDebounceTime$0(ScheduleAfterDebounceTime.java:89) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748)
Attachments
Issue Links
- relates to
-
SAMZA-1064 Standalone Samza with Zookeeper for Coordination
-
- Open
-
- links to
