Details
-
Task
-
Status: Closed
-
Major
-
Resolution: Duplicate
-
None
-
None
-
None
Description
Resource aware schedulers used in Storm optimize on the use of compute, memory and network usage by concentrating executors on as few nodes as possible.
In some instances, this may not be desirable. In cases where the cluster is not shared on when the user explicitly wants the load to be evenly distributed amongst supervisors that have been pre-allocated (for example in a kubernetes environment (AWS EKS or GKE), storm should provide a scheduler to distribute the load evenly, with an optional limit on the number of supervisors that the load will be distributed on - default value will be to use all supervisors.
Since a worker runs only one topology, using all workers for a topology will essentially prevent any other topology from running in that cluster. Proper configuration is therefore required when using Round Robin strategy.